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Smarter technology for a Smarter Planet: 


Finding meaning 
in the noise. 


An unprecedented amount of information flows through companies every 
day. But to what effect? A recent study found that 52% of managers have 
no confidence in the information they rely on to do their job. And 42% of 
them actually use the wrong information at least once a week. Without 
the right approach to business intelligence, companies struggle to turn all 
that information into sound decisions. 


IBM business intelligence and performance management solutions give 
you the smarter tools you need to access the right information, making 
it available to the right people when and how they need it. Today IBM 
is helping over 20,000 companies spot trends, mitigate risk and make 
better decisions, faster. In fact, we helped a major retail supplier achieve 
this by cutting their average financial reporting time by almost 50%. 


А smarter business needs smarter software, systems and services. 
Let's build a smarter planet. ibm.com/intelligence 


IBM, the IBM logo, ibm.com, Smarter Planet and the planet icon are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other 
product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at www.ibm.com/legal/copytrade.shtml. 


"Red Gate software is really top notch. 
It oozes the understanding of what 
people operating on databases really 


need to get their jobs done." 
CTO, com.on AG 


SQL Compare Pro 
Compares and synchronizes SQL 
database schemas. 


SQL Prompt Pro 
— The fastest way to work with SQL. 


All these tools are part of the SQL Toolbelt. 
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SQL Data Compare Pro 
Compares and synchronizes SQL 
database content. 


SQL Backup Pro 
Compresses and encrypts MS SQL 
Server backups. 


To find out more call us free on 1 866 997 0379 or visit www.red-gate.com 


Environment 


—William Sheldon 


Locking down your database environment can be a 
difficult task, but it doesn’t have to be. Learn how 10 


protect your databases by using service accounts and 
encryption to secure your database files and network . 
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Calculating Concurrent 
Sessions, Part 1 

—Itzik Ben-Gan 

Itzik discusses a set-based and a cursor-based solution 

to a T-SQL concurrent sessions querying problem, as well 
as the performance and scaling ramifications of both 
solutions. 


SQL Server Reporting Services 
Questions Answered 


—Tyler Chessman 
Use these techniques and sample reports to get answers to 
your most frequently asked reporting questions. 


Enforce Multitable Constraints 
Using Indexed Views 

—Aviv Zucker 

If you need to enforce business rules on multiple tables, 
try indexed views, which let you join tables and enforce 
uniqueness on the result set. 
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Pivoting the Dynamic Way 
—Mitchel Sellers 

The PIVOT statement is a great tool for presenting data 
in an easy-to-read format, but it has one key limitation: 
You must know about and include all the column names 
in the PIVOT query. You can work around this limitation 
with dynamic pivoting. 


Time to Defrag Your Indexes? 


—Andrew J. Kelly 
When is the right time to defrag? Ultimately, the answer 1s 
up to you, but keep these considerations in mind. 


Editor's Tip 


We'll be looking for you at SQL Connec- 

tions in Las Vegas November 9-12. Meet 

your favorite SQL Mag authors, many of 

whom present SQL Connections sessions. 

Also, drop by the SQL Mag booth and talk with an 

editor! Were always looking for your feedback. For more 

about SQL Connections, visit www.devconnections.com. 
—Sheila Molnar, executive editor 
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Smart Systems Management 


Space and time are not conditions in which we live: 
they are simply modes in which we think." 


Performance Management Resource Management 


- Albert Einstein 


"Don't | know it!" 


- Denny Cherry, 
SOL Server MVP, DBA 
and LiteSpeed® with Fast Compression user 


Save Space, Time and Money 


It's Not Quantum Physics. It's LiteSpeed® for SQL Server's 40:1 
Fast Compression Technology. 


"Using LiteSpeed's new Fast Compression technology is like Quest giving me extra storage 
space and time; said Denny Cherry. "Not only does it store 10 times the number of backups on 
disk, but the process is done in a fraction of the time it took me using traditional compression. 
Now, my recovery operations are fast and easy! It also does the ‘heavy lifting’ for me by 
examining every database every night, to make the best backup decisions based on change 
rates, day of the week and my retention needs. So, l'm no longer wasting my time manually 
checking everything. 


"Bottom line? | don't have to worry about storage space or backup and recovery time. It's the 
coolest gift Quest could give someone in charge of SQL Server backups" 


LiteSpeed’ with Fast Compression vs. Other Tools 
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Backup Time (Min) 
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aditiona iteSpeed 
Compression Tools Fast Compression 


Wi =Total Backup Size (GB) Wl = Total Backup Time (Min) 


Learn more. Read "How to Achieve 40:1 Backup Compression with 
LiteSpeed” for SOL Server's Fast Compression” at www.quest.com/40to1 
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Database Growth Tracker 


—Kevin Kline 
Track database growth across multiple SQL Server instances using this 
CLR stored procedure. 
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PASS Summit Unite 2009 is finally here! Be sure to take advan- 
tage of the 14 full-day pre/post-conference seminars on Novem- 
ber 2 and 6; best practices sessions by the Microsoft SQLCAT 
team; Microsoft CSS engineers, MVPs, and other SQL Server ex- 


perts in the “Ask the Experts” lounge; and face-to-face trouble- 
shooting and project planning support at the CSS and SQLCAT 
SQL Server First Aid Clinic. You can find out more about what 
PASS has to offer at summit2009.sqlpass.org. 
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The Inevitable Virtualization I5 


of SQL Server 


irtualization is certainly no stranger to the 

data center. Nowadays most companies use 
virtualization for test environments, and increas- 
ingly companies use virtualization for production- 
level server consolidation. Even though virtual- 
ization has been used for production workloads 
ranging from file serving to web hosting, there’s 
one application that businesses have been slow 
to virtualize—SQL Server. There’s a perception 
that virtualization can’t provide the performance 
required for SQL Server and other database work- 
loads, which tend to be very I/O intensive. How- 
ever, that perception hasn’t kept pace with today’s 
virtualization technology. 


VMware Performance 
Enhancements 

At the recent VMworld 2009 conference VMware 
CEO Paul Maritz outlined the benefits of VM- 
ware’s new vSphere platform. From the DBA’s 
standpoint some of the most important enhance- 
ments are in performance. The ESX Server 4.0 
platform supports virtual machines (VMs) with 
up to 8 virtual CPUs and up to 255GB of RAM. 
These numbers are significantly higher than ESX 
Server 3.5’s maximums of 4 virtual CPUs and 
64GB of RAM per VM. Even these older levels 
were more than adequate for most SQL Server 
installations. 

Perhaps more important are the enhancements 
to network and I/O performance. ESX Server 
4.0 manages 30GB per second of network traf- 
fic and 350,000 IOPS. While this requires signifi- 
cant hardware resources, it also shows that VMs 
can now scale to meet mainframe-size workload 
demands. 


Hyper-V Scalability Improvements 

Microsoft’s Hyper-V is no slouch in the area of 
scalability either. Hyper-V, especially with Win- 
dows Server 2008 R2, offers many of the same 
scalability benefits as VMware’s ESX Server. Hy- 
per-V R2 supports a maximum of 4 virtual CPUs 
per VM and up to 255GB of RAM per VM. While 
I haven't seen official Microsoft numbers about 
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total IOPS, results posted by QLogic at www 
.qlogic.com/promos/products/hyper-v.aspx show 
that Hyper-V managed 200,000 IOPS. While this 
benchmark may not represent most production 
environments, it does show that Hyper-V is ca- 
pable of supporting very high levels of I/O. For 
best practices and tips about running SQL Server 
in a virtual environment Microsoft has produced a 
detailed white paper, “Running SQL Server 2008 
in a Hyper-V Environment,” at sqlcat.com/white 
papers/archive/2008/10/03/running-sql-server- 
2008-in-a-hyper-v-environment-best-practices- 
and-performance-recommendations.aspx. 


Virtualizing Database Workloads 
Despite the resistance to virtualizing database 
workloads, SQL Server can be an ideal candidate 
for virtualization. For example, why not virtualize 
SQL Server installations in underutilized depart- 
mental or branch office systems that support only 
a few dozen users. So, if virtualization is capable 
of meeting the performance requirements of a 
production database implementation, what are 
the benefits? The two most obvious benefits are 
increased server utilization and ease of manage- 
ment. Virtualization allows you to combine the 
workloads running on many servers and run the 
workloads on far fewer servers. The fewer serv- 
ers you have, the easier your management tasks 
will be. In addition, virtualization makes 
it possible to dynamically adjust VM 
attributes like hot adding RAM or 
disks to a running virtual server in re- 
sponse to increased workload. Virtu- 
alization also enables you to take advan- 
tage of features such as Windows Server 
2008 R2 Live Migration or ESX VMotion, 
which allow you to increase the availability of the 
database services that IT can provide to end users. 
Virtualization might not be required for all instal- 
lations, but it can handle the most demanding 
workloads, and the benefits it offers will continue 
to drive its adoption for all IT services—including 
SQL Server. [SQL | 
InstantDoc ID 102784 


Michael Otey 


(motey @ sqlmag.com) is technical director 
for Windows IT Pro and SQL Server Maga- 
zine and author of Microsoft SQL Server 

2008 New Features (Osborne/McGraw-Hill). 
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Smarter technology for a Smarter Planet: 


Is your information 
withholding information? 


Most businesses have a data management strategy. And another 
data management strategy. And another data management strategy. 
One for every application: ERP, CRM, SCM, HRM, etc. The result is a 
proliferation of siloed data and disjointed information that gets in the 
way of smart decisions. 


An Information Agenda from IBM moves you from an application- 
centric approach to your information toward a broader, more holistic 
view of all of your information systems. So you can make use of your 
data to make decisions faster and with greater confidence. This insight 
can help you optimize your processes, predict market changes and 
turn your information into a strategic asset. Banks can better manage 
financial risk. Retail companies can crystallize trends. Manufacturing 
companies can speed delivery across a complex supply chain. It’s a 
way to make information work for you, instead of vice versa. 


A smarter business needs smarter software, systems and services. 
Let's build a smarter planet. ibm.com/infoagenda 
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Eli Leiba 


ORE on the WEB 


Download the code at 
InstantDoc IDs 102744 


Simple String Encryption and Decryption 


E ncrypting and decrypting strings in T-SQL code 
is complicated in SQL Server 2000 and earlier. 
You have to use the undocumented PWDEncrypt 
and PWDCompare functions or use symmetric 
(1.е., secret) or asymmetric (i.e., public) keys. Fortu- 
nately, encrypting and decrypting strings in T-SQL 
code is much easier in SQL Server 2008 and SQL 
Server 2005. One of the easiest ways is to use the 
ENCRYPTBYPASSPHRASE and DECRYPTBY- 
PASSPHRASE functions. 

To demonstrate the ease of using 
ENCRYPTBYPASSPHRASE and 
DECRYPTBYPASSPHRASE, I created 
and 102788. two simple functions: dbo. Encrypt and dbo 

.Decrypt. The dbo.Encrypt function encrypts the 


LISTING 1: The dbo.Encrypt Function 


USE Northwind 


GO 


CREATE FUNCTION dbo.Encrypt (@str пуагсһаг(4000)) 
Returns varbinary (8989) 


AS 
BEGIN 


DECLARE Gres varbinary(8000) 
SET @res = ENCRYPTBYPASSPHRASE('SQL SERVER 2008" ,@str) 
RETURN (Gres) 


END 
GO 


Q Editor's Tip 


Share your SQL Server 
code, discoveries, and 
solutions to problems. 
Email your contributions 
to r2r@ sqlmag.com. 
Please indude your full 
name and phone number. 
We edit submissions for 
style, grammar, and length. 
If we print your submis- 
sion, you'll get $100. 
—Karen Bemowski, 
senior editor 
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plaintext that you pass to it, whereas the dbo 
.Decrypt function decrypts the ciphertext you pass 
in. 

As Listing 1 shows, the dbo.Encrypt function 
gets a nvarchar string and encrypts it to a varbinary 
result using the ENCRYPTBYPASSPHRASE func- 
tion. ENCRYPTBYPASSPHRASE has two manda- 
tory arguments: a passphrase (which will be used 
to generate the encryption key) and the plaintext to 
be encrypted. In this case, the passphrase is 'SQL 
SERVER 2008' and the @str variable contains the 
plaintext to be encrypted. As the latter demonstrates, 
you have the option of storing either of these argu- 
ments in variables. 

Listing 2 shows the dbo.Decrypt function. It gets an 


LISTING 2: The dbo.Decrypt Function 


USE Northwind 
GO 


CREATE FUNCTION dbo.Decrypt (@епсгур уагВіпагу(8000)) 


RETURNS nvarchar (4999) 
AS 
BEGIN 
DECLARE Gres nvarchar (4999) 
-- BEGIN CALLOUT A 


SET Gres = DECRYPTBYPASSPHRASE ("501 SERVER 2008" ,бепсгур) 


-- END CALLOUT A 
RETURN (Gres) 

END 

GO 


encrypted varbinary string and decrypts it into a nvar- 
char string using the DECRYPTBYPASSPHRASE 
function. Like the ENCRYPTBYPASSPHRASE 
function, the DECRYPTBYPASSPHRASE function 
has two mandatory arguments: a passphrase (which 
will be used to generate the decryption key) and the 
ciphertext to be decrypted. In this case, the passphrase 
is 'SQL SERVER 2008 and the @encryp variable con- 
tains the ciphertext to be decrypted. (The passphrase 
that you use to decrypt the ciphertext needs to be the 
same as the passphrase you used to encrypt it.) 

To call the dbo.Encrypt function, you follow the 
Syntax 


dbo.Encrypt (ItemToEncrypt) 


where ItemToEncrypt is the plaintext you want to 
encrypt or a variable that contains the plaintext. 

To call the dbo.Encrypt function, you follow the 
syntax 


dbo.Encrypt (ItemToDecrypt) 


where JtemToDecrypt is the ciphertext you want to 
decrypt or a variable that contains the ciphertext. 

Listing 3 gives a simple example of how to 
incorporate the dbo.Encrypt and dbo.Decrypt calls 
in code. This code uses the dbo.Encrypt function to 
encrypt the phrase 'Eli Leiba is a SOL guru', stores 
the result (an encrypted string) in the @code variable, 
then prints it. Next, the code uses the dbo.Decrypt 
function to decrypt the string in @code, stores the 
result (1.e., 'Eli Leiba is a SQL guru!) in the (gx vari- 
able, then prints it. 

As you can see, the ENCRYPTBYPASSPHRASE 
and DECRYPTBYPASSPHRASE functions are 
easy to use when you need to encrypt and decrypt 
strings. However, there's а caveat—you'e respon- 
sible for managing and securing the passphrase. By 
default, no permissions are required to execute the 
ENCRYPTBYPASSPHRASE and DECRYPTBY- 
PASSPHRASE functions and you 
don’t have the ability to select the 
algorithm they use for encryption and 
decryption. So, depending on how 
your SQL Server environment is set 
up, a few people might be able to see 
the passphrase and even execute the 
dbo.Encrypt and dbo.Decrypt func- 
tions if they had access to their code. 

However, there are ways to help 
secure passphrases, such as storing 
them in a table that resides in msdb or 
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a master system database and allowing only DBAs 
with the sysadmin server role access to that table. For 
example, the code in Listing 4 creates a table named 
passPhraseEncoders and inserts the 'SQL SERVER 
2008' passphrase into it. After you give this table the 
necessary permissions, you need to create a synonym 
named passPhraseEncoders for master.dbo.passPhra- 
seEncoders in the application database. 

Then, you need to adapt the dbo.Encrypt and dbo 
.Decrypt functions accordingly. In the dbo.Encrypt 
function in Listing 1, you need to replace the line in 
callout A with the code 


DECLARE @passPhrase varchar(80) 
SELECT @passPhrase = 
Pass_Phrase from passPhraseEncoders 
SET @res = EncryptByPassPhrase 
(@passPhrase,@str) 


In the dbo.Decrypt function in Listing 2, you need to 
replace the line in callout A with the code 


DECLARE @passPhrase уагсһаг(80) 
SELECT @passPhrase = 
Pass Phrase from passPhraseEncoders 
SET @res = DecryptByPassPhrase 
(@passPhrase ,@encryp) 


Although storing the passphrases in a protected 
table provides an additional layer of security, even 
this setup has its vulnerabilities. Only reliable DBAs 
and trusted individuals should create these functions. 
It’s also a good practice to create them with the 


LISTING 3: Sample Code Showing How 


to Use the Functions 


USE Northwind 
GO 


DECLARE @x пуагсһаг(4000) 

DECLARE @code varbinary (8000) 

SET @code = dbo.Encrypt ('Eli Leiba is a SQL guru') 
PRINT (соде 

SET @x = dbo.decrypt (@code) 

PRINT @х 


LISTING 4: Code That Creates and Loads 


the passPhraseEncoders Table 


USE master 


CREATE TABLE passPhraseEncoders (Pass Phrase уагсһаг(80)) 


GO 
INSERT INTO passPhraseEncoders values ('SQL SERVER 2008") 


GO 


ENCRYPTION clause. Consequently, you might not 

want to use the dbo.Encrypt and dbo.Decrypt func- 

tions for encrypting text just before exporting it out of 
your database to an exposed text file, for example. 

The dbo.Encrypt and dbo.Decrypt functions work 

on SQL Server 2008 and SQL Server 2005. (Гуе run 

them on SQL Server 2005 SP1 and SQL Server 2008 

Express.) You can download the functions' code by 

going to www.sqlmag.com, entering 102788 in the 

InstantDoc ID text box, and clicking the 102788.zip 

hotlink. 

— Hli Leiba, senior application DBA, 

Israel Electric, and 

senior database consultant, 

2Cher IT Professionals 

InstantDoc ID 102788 


T-SQL Statement Tracks Transaction-Less 


Dates 
M company provides pricing software and 
Y services for retailers and distributors in 
the consumer goods industry. One of the company 
engineers recently asked me to write a T-SQL state- 
ment that he could use to find out which stores didn’t 
have any transactions during a one-week period. 
However, the data in the SQL Server table included 
only the IDs of the stores and the dates on which 
each store had a transaction. So, I was faced with the 
mind-challenging task of writing a T-SQL statement 
that would produce data that wasn’t in the underlying 
table. 

To demonstrate how the T-SQL statement I came 
up with works, I created the sample data shown in 
Table 1, page 12. As you can see, it includes a week’s 
worth of transaction dates (November 1-7, 2009) for 
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five stores, which have the IDs of 100, 200, 300, 400, 
and 500. Only Store 500 has transaction dates for all 
seven days. 

Listing 5, page 12, shows the T-SQL statement, 
GetNonTransactionDaysForStores.sql, I devised. 
This code begins by using the FROM clause in callout 
A to produce two derived tables. This subquery first 
retrieves the unique Store_ID values from MySal- 
esTable and puts them in a derived table named st1. If 
every store had a transaction on at least one day, the 
subquery returns every store ID once. The subquery 
then retrieves the unique TransactionDate values from 
MySalesTable and puts them in a derived table named 
st2. If a store has transactions on all seven days, the 
subquery returns all seven transaction dates. 

The code in callout B performs a Cartesian 


Saravanan 
Radhakrishnan 


November 2009 


TABLE |: Data in the 


MySalesTable Table 
Store_ID TransactionDate 
100 2009-10-05 00:00:00.000 
200 2009-10-05 00:00:00.000 
200 2009-10-06 00:00:00.000 
300 2009-10-01 00:00:00.000 
300 2009-10-07 00:00:00.000 
400 2009-10-04 00:00:00.000 
400 2009-10-06 00:00:00.000 
500 2009-10-01 00:00:00.000 
500 2009-10-02 00:00:00.000 
500 2009-10-03 00:00:00.000 
500 2009-10-04 00:00:00.000 
500 2009-10-05 00:00:00.000 
500 2009-10-06 00:00:00.000 
500 2009-10-07 00:00:00.000 


LISTING 5: GetNonTransactionDaysForStores.sql 


SELECT st1.Store_ID, st2.NoTransactionDate 


product operation to 
match all rows in stl 
with all rows in st2 in 
order to get every pos- 
sible store-transaction 
date combination. The 
results in the Cartesian 
product (st3) are then 
filtered. Because we're 
interested in only those 
dates on which there 
weren't any transactions, 
the code looks for dates 
that are in st3 but not 
st2 for each store. Note 
that if none of the stores 
have a transaction on a 
certain day, the query 
wont return any result 
for that particular day 
for all the stores. The 


(А) FROM (SELECT DISTINCT Store ID FROM MySalesTable (NOLOCK)) AS 511, 
(SELECT DISTINCT TransactionDate AS NoTransactionDate 
FROM MySalesTable (NOLOCK)) AS st2 


(B) WHERE st2.NoTransactionDate NOT IN 
(SELECT DISTINCT st3.TransactionDate 


FROM MySalesTable st3 (NOLOCK) 
WHERE st3.store id = 5%1.5%оге id) 


ORDER BY 5%1.5%0ге id, st2.NoTransactionDate 


GO 


Karen 
Bemowski 
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query assumes that at least one store will have a trans- 
action on any given day in the specified date range. 

Table 2 shows the results from running GetNon- 
TransactionDaysForStores.sql. As you can see, it 
displays only those stores that didn't have transactions 
(which is why store 500 isn't included) and the dates 
of those transaction-less days. 

You can download GetNonTransactionDaysFor- 
Stores.sql and the script I used to create and populate 


TABLE 2: Results from 


GetNonTransaction- 
DaysForStores.sql 

Store ID NoTransactionDate 

100 2009-10-01 00:00:00.000 
100 2009-10-02 00:00:00.000 
100 2009-10-03 00:00:00.000 
100 2009-10-04 00:00:00.000 
100 2009-10-02 00:00:00.000 
100 2009-10-03 00:00:00.000 
200 2009-10-04 00:00:00.000 
200 2009-10-07 00:00:00.000 
200 2009-10-02 00:00:00.000 
200 2009-10-03 00:00:00.000 
200 2009-10-04 00:00:00.000 
300 2009-10-05 00:00:00.000 
300 2009-10-06 00:00:00.000 
300 2009-10-07 00:00:00.000 
300 2009-10-05 00:00:00.000 
300 2009-10-06 00:00:00.000 
400 2009-10-01 00:00:00.000 
400 2009-10-02 00:00:00.000 
400 2009-10-03 00:00:00.000 
400 2009-10-05 00:00:00.000 
400 2009-10-07 00:00:00.000 


the sample table (MySalesTable.Table.sql) from the 
SQL Server Magazine website. (Go to www.sqimag 
.com, enter 102744 in the InstantDoc ID text box, 
and click the 102744.zip hotlink.) Note that because 
this code uses Cartesian product operation between 
the stl and st2 tables, this solution doesn't scale well 
when you have more than a few thousand records to 
consider. 
—Saravanan Radhakrishnan, senior production 
DBA, Revionics 
InstantDoc ID 102744 


The Great Debate: Should You Shrink 


Databases? 


ust like the debate over whether the chicken or the 
ү came first, the debate over whether you should 
or shouldn't shrink databases is still going strong. This 
debate took center stage when the Reader to Reader 
article “Utilities Assess Data-File Usage and Provide 
Commands So You Can Do Something About It” 
(August 2009, InstantDoc ID 102295) was published. 
In this article, David Paul Giroux discussed how to 


use two T-SQL utilities (Candidate Commands and 
Candidate Commands Plus) to manage the size of 
data files. These utilities provide not only information 
about disk and data-file usage but also commands you 
can use to shrink or grow the size of data files. 

In response to “Utilities Assess Data-File Usage 
and Provide Commands So You Can Do Something 
About It,” readers shared their views on the practice 
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of shrinking databases in the article’s reader comment 
area, on SOL Server Magazine's Twitter page (twitter 
.com/sqlservermag), and in their own blogs. We want 
to share some of those comments with you in hope of 
getting a discussion going on whether shrinking data- 
bases is acceptable, and if so, when it's okay to do so. 
If you'd like to share your thoughts, go to www.sqlmag 
.com/Articles/ArticleID/102796/102796.html and 
enter your comments. 


Really Dangerous Advice 

Reader jsegarra was the first to comment on the 
article. Believing that shrinking databases isn't a good 
idea, he commented: 


“T’m sorry but you really shouldn't be shrinking 
databases in the first place. No offense." 


Brent Ozar (brento) echoed that sentiment when he 
wrote, 


"This is really, really, REALLY dangerous 
advice that can cause performance problems. 
I responded to it in full here: http://www.brent 
ozar.com/archive/2009/08/stop-shrinking-your- 
database-files-seriously-now/” 


Note that the item Ozar is referencing is his blog, 
“Stop Shrinking Your Database Files. Seriously. 
Now.” In that blog, he writes, 


“Don’t shrink your database files just to free up 
drive space. Stop. It’s an unbelievably, disgust- 
ingly, repulsively bad idea. Your disk drive space 
is for files, not for ornamentation. You don’t 
get bonused based on the amount of free space 
on your drives. Empty files don’t take longer to 
back up. And so help me, if you find yourself 
shrinking databases so often that you have to 
automate it, you need to cut up your DBA card 
and reconsider your choice of career.” 


Ozar’s blog also includes links to other posts about 
the topic of shrinking databases, including Paul 
Randal’s blog post “Why you should not shrink your 
data files" (www.sglskills.com/blogs/paul/post/Why- 
you-should-not-shrink-your-data-files.aspx.) 


The Script Is Just a Tool 
When readers’ comments started streaming in, we 
asked Giroux for his thoughts. He wrote: 


"The script is a tool. The script does not make 
any changes. The article is narrow in scope and 
gives NO ADVICE. The article is not in regards 
to file management and does not attempt to be 


SQL Server Magazine * www.sqlmag.com 


all encompassing. There are countless articles 
regarding file management, I don't desire to 
write another one. No DBA should attempt to 
read one single article on one single issue and 
expect his/her training to be complete. Learn 
file management elsewhere. As Paul Randal 
states in his post ‘Why you should not shrink 
your data files’: ‘Bottom line - TRY TO AVOID 
running data file shrink at all costs!’ (I added 
the emphasis on ‘try to avoid".) If the compe- 
tent DBA decides a file needs to be shrunk OR 
GROWN, this script can assist the DBA." 


“T created this script because of one particular 
server that had 24 databases and no free disk 
space. One of the databases ran out of database 
free space. I could not unilaterally truncate 
tables, or move databases to a different server or 
add physical drives. The only thing I could do 
instantly was rob Peter to pay Paul. Yes. I shrunk 
the file of Database A so I could increase the file 
for Database B. That gave me time to THEN 
come up with a long term solution." 


After reading Giroux's thoughts, Ozar commented: 


“When you say ‘The article is narrow in scope 
and gives NO ADVICE; it seems to ignore one 
of the early lines in the article: ‘In order to grow 
the data file for DB without depleting all avail- 
able disk space, you must first free up some disk 
space by shrinking other data files." 


“You're telling the DBA that their only option 
- not one of, but their ONLY option - is to 
shrink other data files. That’s bad advice, and 
thats what we're 
trying to avoid. One 
other option might 
be to find out why 
the other data files 
are large, and get 
the right amount of 
disk space to handle 
the problem. Oth- 
erwise, with your 
solution, you're 
recommending that bad id ea." 

they keep growing 

and shrinking different files, which ends up 
being a fragmentation nightmare. If you could 
have included the other options in the article, 
just like you explained them in the comments, 
then the article would have stood great on its 
own. Otherwise, it comes off as an instruction 
manual on how to build a handgun. While some 


“Don’t shrink your 
database files just to 

free up drive space. Stop. 
It's an unbelievably, 
disgustingly, repulsively 
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publications have no problem explaining how to 
build a handgun, my hope is that we strive for a 
higher level of education at SQL Server Maga- 
zine. If we start publishing articles on how to 
truncate tables or how to format drives - without 
explaining the risks and drawbacks - then we're 
doing a bad job of educating the community.” 


Not Ashamed of Being a 
Shrinker 

Also weighing in on the topic was Bill McEvoy. Here’s 
what he had to say: 


“Of all the things to get excited about in the 
MSSQL world, I never thought shrinking 
databases would be one of them. To me, it has 
always been a no-brainer. I’m running out of 
disk space, and I find a database with a ton of 
free space. What do I do? I shrink it! Somebody 
issues a massive update statement that grows 
the transaction log beyond all reason. What do 
I do? I shrink it! Гт а shrinker, and I'm not 
ashamed of it!” 


“Now before you go and accuse me of being a 
‘no-brainer’ myself, hear me out. Pm а DBA 
consultant. I have multiple large clients who have 
hundreds upon hundreds of databases. Most of 
these databases haven't been given the love and 
attention they deserve. As a result, I often inherit 
poorly managed systems and end up spending 
most of my time putting out fires." 


“The single most occurring fire is low disk space. 
Not every organization is blessed with endless 


terabytes of disk storage. One of my clients 
has a large virtual server farm, and each virtual 
server gets just about enough disk space to hold 
the main database and a few backup files. Yes, 
I know, I have more space on my MP3 player, 
but this is the ‘do more with less’ world I work 
in. Unless the DBA team cries for more space, 
we're stuck with what we're given. New servers 
crop up faster than we can document, let alone 
manage. Since adding more disk space requires 
a reboot, we usually have to shrink the data files 
and transaction log in order to keep the server 
up and running.” 


“That being said, a DBA should still exercise 
caution prior to shrinking/truncating a live 
production database. Outages need to be sched- 
uled, backups need to be confirmed, etc. As the 
author stated in his response, sometimes you 
have to do what you have to do. And his script 
helps me do just that." 


“What it comes down to is that if you shrink 
a database, you really need to defragment the 
indexes and data pages. But you are already 
doing that regularly anyway, right?" 


What Do You Think? 
Now that you know how some of your peers feel about 
shrinking databases, how do you feel about it? Go to 
www.sqlmag.com/Articles/ArticleID/102796/102796 
.html and let us know. [SQL 
--Кагеп Bemowski, senior editor, 
SQL Server Magazine 
InstantDoc ID 102796 
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Database Growth Tracker 


Monitor database sizes from a single 


table 


ecause SQL Server is one of the most popular 

database platforms, the number of SQL Serv- 
er instances tends to grow once it has a toehold in 
an enterprise. Many enterprises develop applica- 
tions to run on SQL Server, and SQL Server often 
rides in on the coattails of other Microsoft prod- 
ucts such as SharePoint and Systems Center Oper- 
ations Manager, as well as third-party applications 
purchased for specific needs. This rapid growth 
in the enterprise can make it difficult to properly 
manage all the SQL Server instances in the enter- 
prise in a centralized manner. 

That's where this month's free tool can come 
in handy. The Database Growth Tracker is a cen- 
tralized database growth tracking utility that was 
written by Tara Kizer, a SQL Server DBA and 
Microsoft MVP. (You can read Kizer’s blog at 
weblogs.sqlteam.com/tarad. Kizer offers a vari- 
ety of other interesting custom database main- 
tenance routines at weblogs.sqlteam.com/tarad/ 
archive/2004/07/02/1705.aspx.) The Database 
Growth Tracker lets you centrally query and track 
the amount of space used by databases on as many 
SQL Server instances as you like, without the 
messiness of linked servers. Let’s look at how the 
Database Growth Tracker works. 


Tracking Space Consumption 
This tool uses a CLR stored procedure called 
isp_DatabaseGrowth to query databases for space 
utilization. The CLR stored procedure queries the 
target SQL Server instance and database for space 
consumed using sp_spaceused and sp_databases. 
(Under certain circumstances, such as when a SQL 
Server 2000 database has recently been upgraded 
to SQL Server 2005, the results of sp_spaceused 
might be inaccurate. In those cases, you should 
make sure DBCC UPDATEUSAGE has been 
run recently to ensure accurate row counts.) The 
Database Growth Tracker uses a single table to 
track the SQL Server instances in which isp_ 
DatabaseGrowth will run and retrieve space con- 
sumption information. 

You can collect the information returned by 
isp_DatabaseGrowth daily in a table or a Microsoft 
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Excel spreadsheet to look at trending and the serv- 
er's overall space consumption. Of course, this tool 
is limited to returning the information that's typi- 
cally returned by the system stored procedures that 
it calls, namely sp. spaceused and sp. databases. 


System Requirements 

You can download the Database Growth 
Tracker from Kizers blog at weblogs.sql 
team.com/tarad/archive/2008/12/16/How- 
to-track-database-growth-across-multiple- 
SQL-Server-instances.aspx. (The link to the 
downloadable code is in the first sentence 
of the third paragraph.) On the blog post, 
you'll find both the SQL code that creates 
the server table and the T-SQL code that ex- 
ecutes isp DatabaseGrowth on each target 
SQL Server instance and database. In ad- 
dition, Kizer provides both the C£ source 
code for the CLR object and the compiled 
DLL, as well as a sample SQL script to help 
you get the compiled DLL configured on your cen- 
tral server. Kizer recommends that you regularly 
run isp DatabaseGrowth from a SQL Server Agent 
job. 

The Database Growth Tracker runs on SQL 
Server 2005 and later. The tool otherwise con- 
forms to the requirements of a typical CLR 
object. [SQL 

InstantDoc ID 102370 


DATABASE GROWTH TRACKER 


BENEFITS: Track the growth of your databases 
and forecast their future sizes in a centralized 
location using the Database Growth Tracker. 


SYSTEM REQUIREMENTS: SQL Server 2005 
and later 


HOW TO GET IT: You can download the Database 
Growth Tracker from weblogs.sqlteam.com/tarad/ 
archive/2008/12/16/How-to-track-database- 
growth-across-multiple-SQL-Server-instances 
.aspx. 


Kevin Kline 


(kevin.kline@ quest.com) is the director of 
technology for SQL Server Solutions at Quest 


Software and a founding board member of 
the international PASS. He is the author of 
SQL in a Nutshell, 3rd edition (O'Reilly). 


EDITOR’S NOTE 
We want to hear 
your feedback on 
the Tool Time 
discussion forum 
at sqlforums 
.windowsitpro 
.com/web/forum/ 
categories.aspx? 
catid=169& 
entercat=y. 


November 2009 17 


Securin; J 


William Sheldon 


(wsheldon @ live.com) is a contributing 
editor for SQL Server Magazine, a senior 
software engineer, and a Microsoft 

MVP for Visual Basic .NET. He's also an 

instructor at the University of California, 
San Diego and coauthor of Professional 
Visual Basic .NET 2008 (Wrox). 


18 November 2009 


Environment 


atabase security is a topic that affects 
all DBAs regardless of the size of your 
business or application. If you're read- 
ing this magazine, you need to know how to set up 
a secure database environment. So let's get started. 
I'm not going to spend time on how a developer 
writes secure code or how to add and secure SQL 
Server Reporting Services (SSRS) or other ancillary 
features of your overall data storage solution. Instead, 
ГЇЇ look at security items you should consider on the 
core relational database engine. ГЇЇ approach security 
for SQL Server the way you should approach security 
in general—as a series of layers. First, ГЇЇ show you 
how to secure your SQL Server environment when you 
set up your own database (as opposed to a hosted da- 
tabase) for an application. In future articles, we'll look 
at authorization, authentication, and policy-based se- 
curity settings that are applicable to any database. 
There are certainly differences between the secu- 
rity features available in SQL Server 2008 and SQL 
Server 2005. This article will offer security recom- 
mendations that are valid regardless of version. 


Creating a Secure Environment 
The first element of SQL Server security you need to 
consider is how and where to deploy it. For example, 
a copy of SQL Server installed on the local develop- 
ег laptop probably isn't considered to be secure, re- 
gardless of whether the developer locks down access 
using best practices within the database. If someone 


can access the server's file system and make copies of 
the database files, then it's just a matter of time before 
they crack the security you've applied within those 
files. As a result, you need to consider how to protect 
those files when setting up your server. 

There's certainly ап aspect of physical security 
you should consider—a laptop carried into an airport 
and lost in security is a prime example of not having 
physical security. However, most elements of physical 
security for a server-based database are handled when 
you configure your server in a locked server room. The 
real surface area for a system attack is network-based, 
which is why it's recommended that you place the da- 
tabase on a separate server when setting up a simple 
website. If your website is compromised, in theory, 
any file located on that server might be accessible. So 
if your database is running on the same server as the 
website, the database files could be compromised. 

Ideally, your database server should be a dedicat- 
ed server. Unlike a developer who will cram as many 
different roles and features into a single environment 
as possible, in a production environment, or even in a 
test environment, the goal is for each server to focus 
on a single role. For example, the domain controller 
(DC) shouldn't double as your database server and 
Microsoft Exchange shouldn't be hosted with SQL 
Server. For small-to-midsized businesses (SMBs), 
many of these applications and services might be 
hosted on a single physical machine, while in large 
organizations some of these individual server roles 
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Protect your databases 


by followin ‚ 
security Dest practices 


might be installed across multiple physical machines. 
I won't focus on whether, given your database needs, 
you can successfully host SQL Server within a virtual 
environment, but rather on the fact that by dedicating 
a "machine" to the role of database server you can 
better secure your SQL Server environment. 


Granting Service Account 
Permissions 
The first step in creating a secure environ- 
ment isn't securing just the settings 
used to connect to the database but 
access to SQL Server by the host 
OS. Securing access to your database involves two 
components: the accounts that are used by the server 
and the underlying file system that hosts those files. 
When you install SQL Server, you'll be asked which 
~ accounts should be used to run the database engine, 
not including those associated with additional com- 
ponents such as SQL Server Analysis Services (SSAS) 
and SSRS. These accounts determine the permissions 
associated with the running server. 

During the installation process, you can define 
which services will run and which account or ac- 
counts will be used to define the permissions (autho- 
rization) provided to that service. You can choose 
from three system-defined accounts: Local System, 
Network Service, and Local Service. The Local Sys- 
tem account has highly privileged access to the local 
system and doesn't have access to resources not on 
the local system, thus it would be a poor choice if you 
need to reference data or services on the network. The 
Network Service account has access to network re- 
sources, but only carries the permissions of a typical 
user on the local system. The Local Service account 
is similar to the Network Service account in that it 
has permissions like those granted to a typical user 
on the local system. Unlike the Network Service ac- 
count, however, this account doesn't have permission 
to work with network resources and will pass a null 
account to external network resources. 

So which of these accounts should you use? The 
correct answer is none. Unless you're setting up a 
development environment, you should bypass all 
three of the potential system accounts and instead 
create a user account on the domain or local sys- 
tem. Note that this is true for the relational database 
engine and other subsystems such as SSAS and 
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these 


SSRS. By creating one or more accounts, you can 
customize the exact permissions required and ad- 
just those permissions based on need. Although you 
might initially create a domain account with permis- 
sion to access the local machine, you could change 
permissions later if you needed that account to access 
files on another machine. If that access was needed 
only for a short period of time, you could grant 
and revoke those permissions, and since you would 
only have that account running for the purposes of 
SQL Server, you could be certain you weren't going 
to impact another service. Which is why, of course, 
you'll also want to create a separate account for 
each of the services that are part of your database 
installation. 


Securing Your Database Files 
After creating custom accounts and granting them 
permission to access the file system and folder neces- 
sary to host your database files, you need to protect 
those files. Part of that protection is derived by fol- 
lowing the best practice to associate those files with 
a server that's dedicated to database services. Two 
things to note when you're doing so: First, there are 
several performance-related issues related to competi- 
tion for server resources by different services/applica- 
tions that also make dedicating your server to SQL 
Server a performance best practice. Second, the files 
might be located on a SAN that isn’t physically part 
of the server. In either case, these files are at the core 
of what you're trying to protect from unauthorized 
access. Someone copying these files can essentially 
copy your database and potentially open it on a dif- 
ferent database server. 


Network Security 
Considerations 
There are several things you need to consider regard- 
ing the logical location of the server and database files 
on your network. For SMBs, your network configura- 
tion might just place the production database server 
on a network segment with your internal network. 
However, in larger organizations you'll probably have 
a unique network segment for your production data- 
base servers. 

As Figure 1, page 20, shows, you can place your 
Internet-facing applications on one (or more) net- 
work segments, your internal company on a separate 
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Figure | 


A secure network 
segmentation 
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Public Facing Web and Extranet Applications 


Corporate Data Store 


network segment, and your database server(s) on a 
different network segment. Because your database 
isn’t truly part of your internal network configura- 
tion, you'll want to decide if you're going to use a 
common domain across your internal network and 
the database network. If not, then you're probably 
moving toward a security model that relies on SQL 
Server authentication. Keep in mind that Figure 1 
represents a logical diagram, so although you might 
choose to implement separate firewalls at every loca- 
tion, it's also possible that the two logical firewalls 
surrounding the data servers are in fact a single physi- 
cal device with different rules, depending on the com- 
munication source. 

As you'll also note in Figure 1, there are two con- 
nection points to the Internet. The first connection is 
via web servers, which allow for anonymous access. 
However, your internal network, which is the second 
connection point, contains your corporate users, all 
of whom expect and use network access, not to men- 
tion that they could take a laptop home and connect 
from an external location across a public network. 
In essence, the connection from corporate becomes 
a second attack vector for someone attempting to 
access your database. This scenario, combined with 
the fact that most security attacks originate with in- 
ternal users, makes it easy to justify separating your 
corporate data from your unlimited network access 


by your internal users. 

Once you've decided on your network structure 
you'll need to open a set of ports to access your data 
based on your data access strategy. For example, 
while your website typically works on ports 80 and 
433 for HTTP and HTTPS, SQL Server typically 
uses port 1433 and 1434 for inbound connections. 
Thus, the same tool that was used to exploit some 
unknown IIS vulnerability for your external firewall 
will hopefully be useless in getting through your 
next firewall, which doesn't support traffic to IIS. 
This type of port and protocol shift between your 
external web servers and database provides a layer 
of protection against someone accessing your data- 
base files directly. 


Encrypting Data on the Disk 
One last layer of defense for your database files is 
protecting them on the disk. In SQL Server 2005, it's 
possible to define an encrypted column within your 
tables. Therefore, if you need to store credit card 
numbers or other information that might expose 
people to identity theft, you could add another layer 
of security to that information. This feature is good 
for protecting data but has a few disadvantages. For 
example, it affects performance and requires applica- 
tion changes to ensure the data is actually encrypted 
and decrypted. 
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SECURING SQL SERVER 


SQL Server 2008 Enterprise Edition includes 
Transparent Data Encryption (TDE), which lets you 
specify that the entire database should be encrypted 
on the disk. TDE lets you take an existing database 
and tell SQL Server that the data on disk needs to be 


If someone can 

access the server's file 
system and make copies 
of the database files, 

then it’s just a matter of 
time before they crack 
the security you've applied 
within those files. 


encrypted and no chang- 
es are required from the 
application. The data- 
base engine then handles 
the translation between 
the encrypted data on 
the disk and the decrypt- 
ed data returned to the 
application. It's possible 
to simulate this feature 
using the Encrypting 
File System (EFS) or 
BitLocker Drive En- 
cryption. Keep in mind 
that although BitLocker 
is probably the preferred 


solution for most SQL Server 2005 scenarios, it en- 
crypts only the data “at rest.” Data isn’t encrypted 
when the server is up and running. The process of 
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The Ultimate Log Reading, Auditing Tool 


for SQL Server 


"n SQL Server 2008 and 64-bit support 


setting up database encryption is beyond the scope 
of this article. For more information about TDE, see 
“Database Encryption in SQL Server 2008 Enter- 
prise Edition" (msdn.microsoft.com/en-us/library/ 
cc278098.aspx). 


A Secure Database 
Environment 
Although these practices provide an environment 
that promotes a secure database, they aren't the 
whole answer. For that we need to look at the poli- 
cies that will be enforced within the database server, 
the user accounts, and authorization of those ac- 
counts on the server. Security is a critical facet of 
database management, one that can be defined in 
layers. Those layers can take a logical form in the 
case of multiple network firewalls, database encryp- 
tion, and account permissions. However, they also 
appear when we look at the permissions granted to 
users who access the database or the rules for how 
the database is defined when considering standard 
policies. These additional layers are the focus of the 
follow-up articles on this topic. SQL] 
InstantDoc ID 102775 
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Discover a set-based and a cursor-based solution 


hen you're faced with a T-SQL que- 
rying problem, finding a set-based 
solution (as opposed to a cursor-based 
solution) is recommended for a number of reasons. 
First, set-based solutions are in accord with the rela- 
tional model. Also, they typically require less code 
and less maintenance than cursor-based solutions. In 
addition, set-based solutions tend to perform better. 
For many years, I couldn't find a good-performing 
set-based solution for a particular querying problem 
that involves maximum concurrent sessions. I had 
an adequate set-based solution in terms of its logic, 
but it didn't perform well. The only good-performing 
solution I had was cursor-based. I recently used this 
problem as an exercise in a class I was teaching, and 
one of my students— Darryl Page from the UK—had 
an idea that ultimately led to a good-performing 
set-based solution. In this article, I cover my original 
set-based solution, the cursor-based solution, and 
the performance and scaling ramifications of both 
solutions. Next month ГЇЇ present the improved set- 
based solution based on Darryl’s insight, as well as a 
set-based solution that relies on standard language 
elements that aren't yet implemented in SQL Server 
(as of SQL Server 2008) but hopefully will be sup- 
ported in a future version. 


The Problem 

The task at hand involves a table called Sessions that 
keeps track of user sessions against an application. 
Suppose that every month you need to produce a 
report for the previous month's activity, including the 
maximum number of concurrent sessions for each 
application. That is, for each application, you must 
calculate the maximum number of sessions that were 
active at the same time during the period of interest. 
You might need such information, for example, for 
billing purposes for per-user license types. Use the 
code in Listing 1, page 24, to create the Sessions table 
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in the tempdb database (for test purposes) and popu- 
late it with sample data. 

For simplicity, assume that the Sessions table 
contains only sessions during the period of interest, 
so you don't need to filter a specific date range. Note 
that you should explicitly define the term concurrent 
for your customers. For example, if a session ends at 
the exact point in time that another session starts, are 
the sessions considered concurrent? For the purposes 
of our task, let's assume that two such sessions are not 
considered concurrent. In this case, Table 1, page 24, 
shows the desired output for your solution, given the 
sample data in Listing 1. 

The indexing strategy that would help speed up 
both a set-based and a cursor-based solution is quite 
straightforward. Create one index on the key-list (app, 
starttime, endtime) and another on (app, endtime, 
starttime). Note that you must populate the Sessions 
table with a larger volume of sample data if you want 
to run performance tests. Use the code in Listing 
2, page 24, to create a helper table function called 
GetNums that returns a sequence of integers of a 
requested size. 

Use the code in Listing 3, page 25, to populate 
the Sessions table with a requested number of rows 
by querying the GetNums helper function. The code 
in Listing 3 populates the Sessions table with 10,000 
rows, but you can adjust the number according to 
your needs by assigning your desired value to the 
variable @numrows. The sessions are associated with 
10 different applications using randomization (1 + 
ABS(CHECKSUM(NEWID()) % 10), so about one 
tenth of the rows are associated with each application. 
I didn’t make the number of partitions (applications) 
variable, because adding partitions has a linear effect 
on all solutions (linear complexity), as I will show. A 
more interesting topic for discussion is how changing 
the partition size affects the different solutions’ per- 
formance. Changing the value of @numrows changes 
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CONCURRENT SESSIONS 


LISTING 1: Code to Create and Populate the 


Sessions Table 


SET NOCOUNT ON; 
USE tempdb; 


IF OBJECT ID('dbo.Sessions', 


CREATE TABLE dbo.Sessions 


'U') IS NOT NULL DROP TABLE dbo.Sessions; 


(6 
Кеусо1 INT NOT NULL, 
app УАКСНАК(10) NOT NULL, 
usr УАКСНАК(10) NOT NULL, 
host УАКСНАК(10) NOT NULL, 
starttime DATETIME NOT NULL, 
endtime DATETIME NOT NULL, 


CONSTRAINT PK Sessions PRIMARY KEY(keycol), 
CHECK(endtime » starttime) 


2; 
GO 


CREATE INDEX idx nc app st et ON dbo.Sessions(app, starttime, endtime); 
CREATE INDEX idx nc app et st ON dbo.Sessions(app, endtime, starttime); 


INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(2, "арр1", 'userl', 'hostl', '20090212 08:30", '20090212 10:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(3, "арр1", 'user2', 'hostl', "20090212 08:30", '20090212 08:45"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUESC5, 'арр1', 'user3', 'host2', '20090212 09:00", '20090212 09:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(7, "арр1", 'user4', 'host2', "20090212 09:15", '20090212 10:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(11, 'appl', 'user5', 'host3', '20090212 09:15", "20090212 09:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(13, 'appl', 'user6', 'host3', '20090212 10:30", '20090212 14:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(17, 'appl', 'user7', 'host4', '20090212 10:45', '20090212 11:30'); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(19, "арр1", 'user8', 'host4', "20090212 11:00", '20090212 12:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(23, 'app2', 'user8', 'hostl', '20090212 08:30", '20090212 08:45"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(29, 'app2', 'user7', 'hostl', '20090212 09:00", '20090212 09:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(31, 'app2', 'user6', 'host2', "20090212 11:45", '20090212 12:00"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(37, 'app2', 'user5', 'host2', '20090212 12:30", '20090212 14:00"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(41, 'app2', 'user4', 'host3', '20090212 12:45", "20090212 13:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(43, 'app2', 'user3', 'host3', "20090212 13:00", '20090212 14:00"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(47, 'app2', 'user2', 'host4', '20090212 14:00", '20090212 16:30"); 
INSERT INTO dbo.Sessions(keycol, app, usr, host, starttime, endtime) 
VALUES(53, 'app2', 'userl', 'host4', '20090212 15:30", '20090212 17:00"); 
GO 
LISTING 2: Code to Create the GetNums 
Helper Function 
IF OBJECT ID('dbo.GetNums', 'IF') IS NOT NULL 
DROP FUNCTION dbo.GetNums; 
GO 
CREATE FUNCTION dbo.GetNums(@n AS BIGINT) RETURNS TABLE 
AS 
RETURN 
WITH 
LØ | ASCSELECT 1 AS c UNION ALL SELECT 1), 
L1 А5(5ЕГЕСТ 1 AS c FROM LØ AS A CROSS JOIN LØ AS В), 
(Lz AS(SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 AS B), 
L3 AS(SELECT 1 AS c FROM L2 AS A CROSS JOIN L2 AS B), 
L4 AS(SELECT 1 AS c FROM L3 AS A CROSS JOIN L3 AS B), 
L5 AS(SELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B), 
Nums ASCSELECT ROW NUMBER() OVER(ORDER BY (SELECT 0)) AS n FROM 15) 
SELECT TOP(@n) n FROM Nums ORDER BY n; 
GO 
24 November 2009 


TABLE 1: Desired Output 
from Solution 


app mx 
appl 4 
app2 3 


the partition size, because the number of partitions 
remains constant (10). 

Note that the code in Listing 3 uses randomiza- 
tion to generate sessions that start during the month 
of January 2009 and that last up to 20 minutes. 
It’s important that you not produce sample data 
by merely copying a small set of rows many times, 
because the optimizer sometimes uses methods that 
reuse previously calculated results when the value for 
which the calculation is performed doesn’t change. 
Therefore, a solution’s performance that is calculated 
with such sample data might not adequately reflect 
your actual system’s performance. 


Original Set-Based Solution 
Listing 4 shows the original set-based solution 
that I used to address the problem. The main idea 
behind this solution is that the number of concurrent 
sessions changes only when a session either starts 
or ends. Between such events, the number of concur- 
rent sessions doesn’t change. Moreover, you can con- 
sider the start of a session as an event that increases 
the count of concurrent sessions and the end of a 
session as an event that decreases the count. There- 
fore, the maximum number of concurrent sessions 
must take place at a point when one of the sessions 
starts. 

The solution in Listing 4 starts by defining a 
common table expression (CTE) called AppTime- 
stamps that returns the app and starttime attributes 
from all sessions. Then the code defines a CTE called 
Counts based on a query against AppTimestamps. 
The query against AppTimestamps invokes a scalar 
subquery expression aliased as cnt that for each outer 
row counts the number of concurrent sessions for 
the current application, at the current timestamp. 
Remember that in this case were not considering 
sessions that end at the given timestamp. Therefore, 
the predicate in the subquery isn’t T.ts BETWEEN 
S.starttime AND S.endtime; instead, its T.ts >= 
S.starttime AND T.ts < S.endtime. Finally, the outer 
query groups the rows from the CTE called Counts 
by application, then returns the maximum count for 
each application. 

This solution is logically sound, but it's quite slow 
when you run it with large partitions. In addition, rather 
than scaling linearly, it scales in a quadratic manner (№). 
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Examining the execution plan for this solution, which 
Figure 1 shows, reveals the performance problem. 

Г] use the letters p and ғ to represent the number 
of partitions (applications) involved and the number 
of rows per partition, respectively. The plan starts 
with a full scan of the index on (app, endtime, start- 
time) to retrieve the application and starttime values 
from all rows (for the CTE AppTimestamps). The 
total number of rows scanned is about p х r. Next, 
for each row returned from the Index Scan operator, 
the plan invokes a seek and partial scan of the index 
on (app, starttime, endtime). This work is represented 
by the Index Seek operator. 

Observe the Seek Predicates (aka SEEK:() predicate) 
and Predicate (aka WHERE:() predicate) in the prop- 
erties of the Index Seek operator. SQL Server Books 
Online explains the difference between the two: “The 
storage engine uses the index to process only those rows 
that satisfy the SEEK:() predicate. It optionally may 
include a WHERE:() predicate, which the storage engine 
will evaluate against all rows that satisfy the SEEK:() 
predicate (it does not use the indexes to do this).” 

This means that all rows satisfying the Seek 
Predicates will be scanned, and SQL Server will then 
examine each row, evaluating the Predicate to deter- 
mine whether or not to return the row in the opera- 
tors output. So in practice, for each row returned 
from the Index Scan operator the Index Seek 
operator will end up scanning all rows where the app 
value is the same as in the outer row, and the starttime 
value is smaller than or equal to the timestamp value 
in the outer row. This means that for each outer row, 
the Index Seek operator will scan at the leaf about r/2 
rows on average. So the total number of rows scanned 
by both the Index Scan and Index Seek operators 18: 
рхг+рхгх (1/2), which is equal to p x r + p X 
r?/2. So, if the number of rows per partition increases 
by a factor of f, the total number of rows processed 
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LISTING 3: Code to Populate the Sessions Table 
with a Large Number of Rows 


SET NOCOUNT ON; 
USE tempdb; 


TRUNCATE TABLE dbo.Sessions; 


DECLARE Gnumrows AS INT; 
SET @numrows = 10000; -- Change this value according to your needs 


INSERT INTO dbo.Sessions WITH(TABLOCK) 
(keycol, app, usr, host, starttime, endtime) 
SELECT 
ROW NUMBER() OVER(ORDER BY (SELECT NULL)) AS keycol, 
ру 
DATEADDC 
second, 
1 + ABSCCHECKSUM(NEWID())) % (20%60), 
starttime) AS endtime 
FROM 


SELECT 
'app' + CAST(1 + ABSCCHECKSUM(NEWID())) % 10 AS VARCHAR(10)) AS app, 
"иѕег1' AS usr, 
'hostl' AS host, 
DATEADD( 
second, 
1 + ABS(CHECKSUM(NEWIDO)) % (30%24%60%60), 
120090101!) AS starttime 
FROM dbo.GetNums(Gnumrows) AS Nums 
WHERE n «- Gnumrows 
) AS D; 
GO 


LISTING 4: Original Set-Based 
Solution 


WITH AppTimestamps AS 
С 
SELECT арр, starttime AS ts FROM dbo.Sessions 
Jis 
Counts AS 
C 
SELECT app, 
(SELECT COUNT(*) 
FROM dbo.Sessions AS S 
WHERE T.app = S.app 
AND T.ts >= S.starttime 
AND T.ts < S.endtime) AS cnt 
FROM AppTimestamps AS T 
) 
SELECT app, MAX(cnt) AS mx 
FROM Counts 
GROUP BY app; 


Dr DE 
3 < а fg Ы; 

— Stream Aggregate ~ Parallelism —Stream Aggregate — Nested Loops 2 Index scan (Nonclustered) 
(Aggregate) (Gather streams) (Aggregate) (inner Join) [sessions]. [idx nc app.et. st] 
Cost: 0 X Cost: 0 X Cost: O € Cost: 0% cost: 1% 

L 
ra 35 ^ 
A = = 
Stream Aggregate Index seek (Nonclustered) 
ол I яе (Aggregate) [sessions]. [idx nc app. st. et].. 
j Cost: 28% Cost: 71 % 
Predicate 


[tempdb].[dbo].[Sessions].[starttime] «[tempdb].[dbo]. 
[Sessions].[endtime] as [S].[endtime] 

Object 

[tempdb].[dbo].[Sessions].[idx nc app. st et] [S] 

Seek Predicates 

Seek Keys[1]: Prefix: [tempdb].[dbo].[Sessions].app = 
Scalar Operator([tempdb].[dbo].[Sessions].[app]), End: 
[tempdb].[dbo].[Sessions].starttime <= Scalar 
Operator([tempdb].[dbo] [Sessions] [starttime]) 


Figure 1 


Execution plan for original set-based solution 
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LISTING 5: Cursor-Based Solution 


DECLARE 


@арр AS VARCHAR(19), @prevapp AS VARCHAR (10), @ts AS datetime, 


Gevent type AS INT, @cnt AS INT, @mx AS INT; 
DECLARE GResult AS TABLE(app VARCHAR(19), mx INT); 
DECLARE C CURSOR FAST FORWARD FOR 


SELECT app, starttime AS ts, 1 AS event type 
FROM dbo.Sessions 


UNION ALL 


SELECT app, endtime, -1 
FROM dbo.Sessions 


ORDER BY app, ts, event type; 
OPEN C; 


FETCH NEXT FROM C INTO барр, @ts, Gevent type; 
SELECT @prevapp = барр, @cnt = Ø, @mx = 0; 


WHILE QGGFETCH STATUS = 0 
BEGIN 
IF @арр <> @prevapp 
BEGIN 
INSERT INTO @Result VALUES(@prevapp, (mx); 
SELECT @prevapp = барр, @cnt = Ø, @mx = 0; 
END 


SET @cnt = @cnt + Gevent type; 
IF @cnt > @mx SET (тх = (спе; 


FETCH NEXT FROM C INTO барр, @ts, @event_type; 
END 


IF @prevapp IS NOT NULL 
INSERT INTO @Result VALUES(@prevapp, @mx) ; 


CLOSE GS 
DEALLOCATE C; 


SELECT * FROM GResult; 
GO 


increases by close to a factor of f°. The rest of the 
work in the execution plan involves calculating the 
aggregates (count, and later the maximum count). 

I provide actual benchmark numbers in a later 
section, but you can see that this solution is quite 
inefficient. I wouldn't use it unless I were dealing with 
very small partitions. 


Cursor-Based Solution 

Listing 5 shows my cursor-based solu- 
tion. This solution also relies on the idea 
that the start of a session increases the 
count of concurrent sessions, whereas 
the end of a session decreases the count. 
The most important part of this solution 
is the query that the cursor is based on: 


SELECT app, starttime AS ts, 1 AS 
event type 
FROM dbo.Sessions 


UNION ALL 


SELECT app, endtime, -1 
FROM dbo.Sessions 


ORDER BY app, ts, event type; 


The query returns all start and end 
events, with a +1 value representing 
a start event type, and a -1 value 
representing an end event type. The 
query orders the rows by application, 
timestamp, and event type. It's obvious 
why you would want to order by appli- 
cation and timestamp. The reason for 
adding the event type (-1 or +1) to the 
ORDER BY list is to ensure that if 
one session ends when another starts 
you won't consider them as concurrent 
(-1 sorts before +1). 

The rest of the code simply scans 
the cursor rows in order, calculating a running total of 
the event type within the current application, which is 
the number of concurrent sessions at that point. The 
code keeps the maximum count in a variable, and when 
It's done processing an application, the code inserts a 
row with the application and maximum count into a 
table variable. Ultimately, the code queries the table 
variable to return the results. 


(concatenation) ' 


35 | 
Merge ЛОП. — Compute Scalar 
Cost: 1 Ж 


E! 


“Compute Scalar 
cost: 1 X 


Cost: 29 X 


Index scan (Nonclustered) 
[sessions]. [idx nc app. st. et] 


Cost: 35 X 


Index Scan (Nonclustered) 
[sessions]. [idx nc app. et. st] 
COSUt: 35 5 


Figure 2 


Execution plan for cursor-based solution 
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Figure 3 


Benchmark test results 


The execution plan in Figure 2 shows the query that 
the cursor-based solution is based on. Remember that 
you created two indexes to support your solutions— 
one on (app, starttime, endtime), and another on 
(app, endtime, starttime). Notice in the plan that both 
indexes are scanned in order, then the plan uses a 
Merge Join (Concatenation) operator to unify the rows 
returned from both Index Scan operators and return 
them in the requested order. This Merge Join operator 
relies on index ordering and therefore avoids the need 
to sort the data. I was surprised when I realized that 
the optimizer could achieve this without adding a Sort 
operator to the plan. This means that for p partitions 
with r rows each, you get a total of 2 X p X r rows 
scanned, and no extra cost due to sorting. Therefore, if 
the number of rows per partition increases by a factor 
of f, the number of rows processed also increases by a 
factor of f. Thus, the cursor-based solution has linear 
scaling. Keep in mind that processing each row with a 
cursor-based solution is more expensive than with a 
set-based solution. However, in our case the set-based 
solution has quadratic complexity, whereas the cursor- 
base solution has linear complexity. 


Benchmark Results 
I ran benchmarks to test my set-based and cursor- 
based solutions. I used sample data with 10 partitions 
and a total number of rows ranging from 10,000 
(1,000 per partition) to 100,000. Figure 3 shows the 
results of these benchmark tests. 

You can clearly see the linear complexity of the 
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Num Rows 


cursor-based solution and the quadratic complexity of 
the set-based solution. This means that this particular 
set-based solution is efficient only compared with the 
cursor-based solution, for very small partitions (about a 
few hundred rows per par- 
tition). The cursor-based 
solution is more effi- 
cient for large partitions, 
because it scales better. 


row with a 
The Best Is Yet. Cursor-based 
To Come 

For years, I pondered a 
better set-based solution 
to the maximum concur- 
rent sessions problem. My 
original set-based solution 
performs inefficiently, espe- 
cially with large partitions. 
My cursor-based solution performs reasonably well and 
scales linearly; however, it still has the basic cursor-based 
solution disadvantages (1.e., it isn’t in accord with the 
relational model, and it has problematic readability and 
maintenance). Next month ГЇЇ cover a new and better- 
performing set-based solution with linear scaling. In 
addition, ГЇЇ discuss a standard solution based on win- 
dowing aggregates that isn’t yet supported in SQL Server 
but hopefully will be in the future. In the meantime, let 
me know if you come up with a good-performing set- 
based solution to the problem. |5121. 
InstantDoc ID 102734 


solution. 


Processing each 


solution is more 
expensive than 
with a set-based 


November 2009 27 


One Place. One Time .. 
DevCowectione 


NOVEMBER 9-12, 2009 е LAS VEGAS, NV 
Mandalay Bay Resort & Casino 


a SERVER MICROSOFT VISUAL 
SQL ASP.NET sTUDIOCS. NET ARCHITECT 


* MICROSOFT DOTNETNUKE 
SharePoint WINDOWS EXCHANGE _ openrorce‘09 
CONNECTIONS 


BONUS: MOBILE CONNECTIONS and CLOUD COMPUTING tracks! 


EE Be among the first to get the insiders scoop 
Exciting, Announcements on the products and technology you rely on! 


As a DevConnections attendee, 

you and your colleagues can 
attend all of the Connections à 4 Я, 
shows, and cross between all 4 
of the sessions, at the same 3 


time for the same price. SCOTT QUENTIN KIMBERLY L. THOMAS 


QN 


E E Nn NIRE FORT com e 
INDUSTRY EXPERTS à E pus ме ышы ы 
огрогате Vice епега! Мападег irector, 
m 200+ IN-DEPTH SESSIONS President, .NET of Database Co-chairperson SharePoint Group 
Developer Division Engine Group 


HOT TECHNOLOGY TOPICS INCLUDE: Silverlight 3 * RIA Services * ASP.NET MVC * SharePoint 2010 
Analysis Services 2008 * Logging and Recovery in SQL Server * SQL Server "Madison" * Windows Azure 
Workflow 4.0 * Visual Studio Team System 2010 * .NET 4.0 Language Features 


>> REGISTER NOW—before Mandalay Bay rooms sell out at the $149 show rate! 


CHECK WEB SITE FOR DESCRIPTIONS OF SESSIONS AND WORKSHOPS 
www.DevConnections.com • 800.438.6720 * 203.268.3204 * Register Today! 


611009 97 


Get тоге out of your reports using these 


techniques and sample 


pplications tend to generate a lot of data— 
A data that end users, developers, and DBAs 

need to turn into information to answer 
questions. SQL Server Reporting Services (SSRS) 
helps address this need. However, SSRS is itself an 
application that generates a lot of data. In this article, 
you'll see how to leverage sample reports and tech- 
niques to answer common questions related to SSRS 
reports. Note that all of the sample reports and tech- 
niques discussed in this article work with both SSRS 
2008 and SSRS 2005. You can download the samples 
for this article by going to www.sqlmag.com, entering 
InstantDoc ID 102645, and clicking the Download the 
Code link at the top of the article page. 


Which Reports Are Being Run, 
How Long Are They Taking, and 
Who Is Viewing Them? 
SSRS logs every report request to an internal table in the 
Report Server database. This logging behavior is docu- 
mented in the *Report Server Execution Log" section 
of SQL Server Books Online (BOL). Direct querying 
of this table, however, isn’t recommended, and the data 
in this table is removed after 60 days by default. Instead, 
Microsoft provides, as part of a larger set of SSRS sam- 
ples, a SQL Server Integration Services (SSIS) package 
(RSReportExecutionLog_Update.dtsx) that extracts 
the log data into a separate database, as well as several 
reports for viewing the log data. You can download 
the SSRS 2008 and 2005 samples from www.codeplex 
.com/MSFTRSProdSamples. Once you've installed the 
samples, navigate to the \Samples\Reporting Services 
Report Samples\Server Management Sample Reports 
folder. ’'m not going to spend time reviewing the SSIS 
package responsible for the data extraction, but you can 
see a diagram of the resulting schema in Figure 1. Let's 
take a look at the sample reports. 

Figure 2, page 30, shows a portion of the Execu- 
tion Summary report. (For space purposes, I removed 
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the graphs that show report executions by day of the 
month and day of the week, as well as a Top 10 Largest 
Reports table.) The Execution Summary lets you see, 
for a given date range, general statistics (e.g., total 
number of executions), a list of the 10 most popular 
reports, a list of the 10 longest running reports, and 
a list of the top 10 users for the SSRS instance. After 
you click a report name in any of the Top 10 tables, 
the Report Summary report, shown in Figure 3, page 
30, is displayed. This report provides, for a given date 
range, detailed report execution information, including 
the number of successful and failed executions, report 
status codes, and a list of parameters used. 

The last sample report, Execution Status Codes, 
summarizes successful and failed report executions, and 
offers a detailed list of all report success/failure counts. 


Tyler Chessman 


(tylerc@ microsoft.com) is a technology spe- 
Galist for Microsoft, helping customers evaluate 
and adopt SQL Server. He's also the founder 
of SQL Server Power (www.sqlserverpower.com), 
a website that provides utilities for SQL Server 


and related business intelligence products. 


Type of report e.g. Live, Cache, 
Snapshot, ete. 


Tracks inserts of data into the 


various 


Figure 1 


Execution log schema 


November 2009 29 


жал | SSRS QUESTIONS ANSWERED 


Start Date [4/1/2009 =) End Date [4/30/2009 a 


4E oft > n [itm] [Find] Next 
Report Execution Summary (4/1/2009 - 4/30/2009) 


Looking back at Figure 1, you might con- 
sider creating additional reports, such as a 
User Detail report, to display a list of the 
reports a particular user has executed for 
a given date range. 


i4 Select a format 


Total No. Executions 42 Successful Executions 42 How Many Report 
Avg. Executions / Day 1 Failed Executions 0 Subscriptions Have 
Been Created and Did 
Top 10 Most Executed Any of Them Fail? 

Report Name Executions It's often useful to know the overall count, 
[TU НЕТ i- status, and cause of errors (if applicable) 
8 with respect to report subscriptions. I 
Employee Sales Summary 2008 6 recently met with a business analyst who 
EmpSalesByTerritory 5 was given the responsibility of moni- 
; toring, troubleshooting, and restarting 
E L A report subscriptions. When she asked me 
ро и uu OS ^ч how to perform this task in SSRS, I real- 
Erecatonsurnaly 1 oe ized a custom report would be needed. 
EEES геп 2008 : Sis (SSRS provides out-of-the-box func- 
Боору uae 8 29 tionality within Report Manager to view 
Product Line Sales 2008 > Б> status information for а single subscrip- 
Top 10 Users tion, but it doesn’t provide an aggregated 

User Name Executions view across multiple subscriptions.) 
SQLOSRTM\Administrator 244 SSRS stores subscription information 
SQLOBRTMASQLSvcAcct 20 in an internal table; additional tables 


are used for the applicable subscription 
schedule(s). Microsoft doesn't officially 
support direct querying of these tables, 
because they can change from version to 
version, but you can query for subscription 


Start Date [4/1/2009 S8 End Date [5/17/2009 ea . . : 
information via the Report Server Web 
Report — [EmpSalesByTerritory zl Service. (Note that SSRS supports Web 
iq 4[ оі» bi [00% :1 | Find | Next [Select а format zl Services as a data source.) 
If you haven't already done so, down- 
Report Summary (4/1/2009 - 5/17/2009) 


load the samples for this article. Then 
open up the SubscriptionReports.rptproj 


Figure 2 


Report Execution Summary 


Report Name EUQSMUSBEPEtOO: project in Business Intelligence Develop- 
Path /AdventureWorks 2008 Sample Reports/EmpSalesByTerritory ment Studio (BIDS). (Га like to say a 
Total No. Executions 9 


special thanks to Lukasz Pawlowski, a 
program manager on Microsoft’s SSRS 
team, for providing these reports.) 

There are two main reports in this 
project: SubscriptionsList (shown in 
Avg. Num Rows 5 Figure 4, with some information removed 
for space purposes) provides a summary 


Successful Executions 9 
Failed Executions 0 
Avg. Executions / Day 0.20 
Avg. Size (Bytes) 4,875 


Status Codes Total Rate И К 

of subscription executions and errors and 

rsSuccess 9 100,0% . : Е . 
a detailed list of subscriptions for a given 
Report Parameters month or year. (This report will return an 
Parameter Name / Value Execution Count error if there isn’t a least one subscription 
Е TerrGroup defined on the server.) I added a table at 
Europe 1 the bottom of the report that displays a 
North America 8 list of any subscriptions that have yet to 
be executed. The other report, Subscrip- 
Figure 3 tionsByExtension, displays a list of all 
Report Summary report subscriptions broken out by extension 
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SSRS QUESTIONS ANSWERED eama | 


i4 4 hi ofi b bi [100% -] 


Executed Time: — 6/4/2009 6:05:59 PM 


Show This Month's Failures 


Subscription Details 


Company Sales 


6/3/2009 10:00:42 PM 


TLCSVRSGQLAOSVAdministrator 


6/4/2009 12:09:04 PM 


Subscription Not Yet Run 


Report 
TylerEmpSales 


Owner 


TLCSVRSOQLAOS Garrett! 


| Find | Next [Select а format z] Export 


а d 


© Show Details By Report 


Mail sent to amyO 


Mail sent to Administrator 


Mail sent to Administrator 


6/4/2009 6:05:59 PM 
TLCSVRSOLAOSVAdrinistrator 
Pagel of 1 


Send e-mail to amy0 


Send e-mail to Administrator 


Send e-mail to Administrator 


Send e-mail to Garrett? 


type (standard or data driven). SubscriptionsByExten- 
sion relies on two subreports, StandardSubscriptions 
Extension and DataDrivenSubscriptionExtension, 
which are also included in the project. 

Even if monitoring subscriptions isn't required in 
your organization, you should review these reports 
because they contain some interesting features, 
including dynamic data connections, the use of XML 
Web Services as a data source, show/hide/visibility 
control, inline code, and subreports used as text boxes 
within a table. 


Where Did This Report Come 
From? 
A report can sometimes lead to further questions 
regarding the validity of the report itself. Consider a 
scenario in which an end user is viewing a report that 
someone else executed and exported to Microsoft 
Excel. Before the user will trust the information in 
the report, he or she might want to know where the 
report came from, when it was run, who ran it, what 
filters were applied, and so on. Many of these ques- 
tions can be addressed simply by taking advantage 
of the built-in fields available in the Report Designer. 
If you look at the page footer in Figure 4, you can 
see I added several fields, including the report name, 
path, and server, as well as the date and user respon- 


SQL Server Magazine * www.sqlmag.com 


sible for executing the report. You should consider 
including this information across all the reports in your 
environment. In the body of the report, you might also 
want to provide a list of any parameter values. In the 
sample code, Гуе provided a blank report template 
that you can reuse in your environment. (Note: ’m 
using the term “template” to simply describe a report 
that other reports can be based upon.) An interesting 
way to make use of templates in BIDS is to place the 
report’s .rdl file in the \Program Files\Microsoft Visual 
Studio 8\Common7\IDE\PrivateAssemblies\Project 
Items\ReportProject subfolder, so the report appears as 
an additional option when adding a new report. 


Use SSRS to Get Answers 
SSRS will likely have multiple “administrators” (e.g., 
DBAs, web administrators, and the developers and 
power users responsible for report generation and 
maintenance) who will have questions related to usage, 
performance, and troubleshooting. By extracting 
information from SSRS, and in some cases querying 
SSRS directly, a lot of information can be provided 
via custom reports. Also, end users might occasionally 
question the validity of reports. SSRS's built-in fields, 
when placed directly within a report, can help address 
many of these questions. SQL] 
InstantDoc ID 102645 


Figure 4 


SubscriptionsList report 
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DEEP 
DIVE, INTO 


МЕТІ аме 


WHEN 


December 10, 2009 


WHERE 


Your computer 


COST 


$99 for all 3 lessons 


LESSONS 


11:00 am ET - VMware Virtualization 
Capabilities and the vSphere Platform 


12:30 pm ET - Deploying and 
Managing vSphere 


2:00 pm ET - High Availability and 
Resource Management with vSphere 


HOW 


Register at www.windowsitpro.com/go/ 


elearning/VMwarevSphere 


Eh john Savill 


Explore the major functionality 
capabilities of the vSphere 
virtualization platform, including 
identification of the changes 
from ESX 3.5. 


Join MVP John Savill on December 10, 2009 for 3 in-depth lessons 
and Q&A sessions on how to ensure that vSphere is deployed and 
maintained in the most optimal way. 


What you'll take-away from this exclusive eLearning series: 


Wi Understanding the different types of virtualization available 
and how they are best suited to your organization 


li Understanding how vSphere is deployed and managed with 
focus on additional capabilities through Virtual Center 


Bi Learning about the high availability capabilities of vSphere 
through vMotion and storage migration capabilities, including 
disaster recovery site capabilities 


INSTRUCTOR: 


John Savill is the author of the popular FAQ for 
Windows and a contributing editor to Windows IT 
Pro. He is an advisory architect for EMC's Microsoft 
consulting practice. He's an MCITP: Enterprise 
Administrator for Windows Server 2008 and a 
10-time MVP. His latest book is The Complete Guide 
to Windows Server 2008 (Addison-Wesley). 


Learn more about the speaker, lessons, 
and how to reserve your seat at: 
www.windowsitpro.com/go/elearning/ 


WindowsITPro 


VMwarevSphere 


Enforce. 


ndexed Views 


Indexed views work better than CHECK 


constraints and triggers 


here are cases in which you need to enforce 
i business rules between multiple tables. One 
such case is when validating that a custom 
sequence, used as a primary key in two tables, isn't used 
more than once. In such cases, CHECK constraints are 
limited to a single table. Triggers can be used but pro- 
vide a non-optimal solution because you need to create 
a trigger on both tables. So how do you implement 
multitable constraints, then? By using indexed views. 


Implementing Indexed Views 
Consider a scenario in which you have an inventory 
control system for managing items. Each item can 
be stored in either a tray or a box, but not in both at 
the same time, obviously. Figure 1 shows a simplified 
schema that holds the data. The ITEM table holds 
all items in the system, and the referencing tables 
ITEM IN BOX and ITEM IN TRAY each hold the 
location of an item, in a box or in a tray, respectively. 
For simplicity, additional columns aren't shown. 

To enforce the business rule that says the same 
ITEM ID can’t be inserted into both ITEM. IN BOX 
and ITEM. IN TRAY, you could add INSTEAD OF 
INSERT and UPDATE triggers to both ITEM IN 
BOX and ITEM IN TRAY tables, validating this 
logic. However, an alternative that's much easier to 
maintain is to use an indexed view. 

Indexed views are typically used for aggregations, 
but work well for enforcing multitable constraints 
because they allow joining tables and enforce unique- 
ness on the result set. Unlike standard views, which 
hold only the underlying SQL query and are replaced 
with this query upon execution, an indexed view is 
materialized with the query's data. This means that 
when the underlying tables’ data is modified, the view 
is updated. When you query the view, all the data has 
already been calculated. You index a view by creating 
a unique clustered index on it. (Indexed views are 
available in SQL Server 2000 and later.) 
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The indexed view will join ITEM ІМ BOX and 
ITEM IN TRAY on ITEM ID. If the same ITEM . 
ID is found in both tables, the ITEM ID will be 
returned by the underlying query of the indexed view. 
Because you want to reject such a case, you need a 
way to duplicate this row more than once so that the 
unique index will fail the insert/update. To do so, you'll 
want to join the two tables to an additional NUMS 
table that has only two rows in it and is used solely for 
multiplying the output by 2 (two rows instead of one 
per violating ITEM. ID). Now the uniqueness of the 
view will reject the duplicated rows. 

The following steps walk you through implementing 
indexed views to enforce multitable constraints: 


ORE on the WEB 


Download the code at 
InstantDoc ID 102690. 


Aviv Zucker 


(aviv.zucker 9) intel.com) is a software and 
database developer at Intel, with more than 
10 years of experience with SQL Server 
and NET. 


ITEM ІМ BOX 


Figure 1 


Sample inventory control system schema 


TRAY ІП 
ITEM ID 
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ENFORCE MULTITABLE CONSTRAINTS 


LISTING I: Code to Create ITEM IN BOX and ITEM IN TRAY 


Tables 


1. Create the ITEM ІМ. 
BOX and ITEM ІМ. 


== ІШЕМ 


CREATE TABLE ITEM 
C 
ITEM ID INT PRIMARY KEY NOT NULL 


GO 


-- ITEM IN BOX 


CREATE TABLE ITEM IN BOX 
C 


BOX ID INT NOT NULL, 
ITEM ID INT NOT NULL 
) 
GO 


ALTER TABLE ITEM IN BOX ADD CONSTRAINT IIB PK PRIMARY KEY CLUSTERED (BOX ID, ITEM ID) 


GO 


ALTER TABLE ITEM IN BOX ADD CONSTRAINT IIB I FK FOREIGN KEY (ITEM ID) REFERENCES ITEM (ITEM ID) 


GO 


-- ITEM IN TRAY 
CREATE TABLE ITEM IN TRAY 
C 


TRAY ID INT NOT NULL, 
ITEM ID INT NOT NULL 
) 
GO 


ALTER TABLE ITEM IN TRAY ADD CONSTRAINT IIT PK PRIMARY KEY CLUSTERED (TRAY ID, ITEM ID) 


GO 


ALTER TABLE ITEM IN TRAY ADD CONSTRAINT IIT I FK FOREIGN KEY (ITEM ID) REFERENCES ITEM (ITEM ID) 


GO 


-- NUMS 


CREATE TABLE NUMS 


NUM INT NOT NULL 


) 
GO 


INSERT INTO NUMSCNUM) VALUES(C1) ; 
INSERT INTO NUMSCNUM) VALUES(2) ; 
GO 


LISTING 2: Code to Create an 
Indexed View 


-- Cleanup before proceeding 
DELETE FROM ITEM IN TRAY; 

DELETE FROM ITEM IN BOX; 
DELETE FROM ITEM; 

GO 


CREATE VIEW ITEM IN TRAY OR BOX V(ITEM ID) 
WITH SCHEMABINDING 
AS 


SELECT IIB.ITEM ID 
FROM dbo.ITEM IN TRAY IIT 
INNER JOIN dbo.ITEM IN BOX IIB 
ON IIT.ITEM ID - IIB.ITEM ID 
CROSS JOIN dbo.NUMS 
WHERE NUMS.NUM «- 2 


GO 
CREATE UNIQUE CLUSTERED INDEX ITEM IN TRAY OR. 


BOX V UI ON ITEM IN TRAY OR BOX V(ITEM ID) 
GO 
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TRAY tables using the 
script in Listing 1. 

2. Use Web Listing 1 (www 
.sqimag.com, InstantDoc 
ID 102690) to insert 
violating data into the 
ITEM_IN_BOX and 
ITEM_IN_TRAY tables 
and identify it. 

3. Create an indexed view 
using Listing 2. 

4. Rerun your inserts using 
Listing 3 to see how the 
indexed view rejects the 
violating row. This view 
will always be empty. 

5. Use Web Listing 2 to 
drop all the objects. 


Enforce 
Multitable 
Constraints 
More Easily 
This common database task 
doesn’t have to be difficult. 
Indexed views and the tech- 
nique for the duplication of 
rows using NUMS tables 
offer a great solution for 
enforcing business rules 
between multiple tables. 
SQL] 
InstantDoc ID 102690 


LISTING 3: Code to Show Indexed View Rejecting 
Violating Row 
INSERT INTO ТТЕМ(ТТЕМ ТО) VALUES (1) 
INSERT INTO ITEM(ITEM ID) VALUES (2) 
INSERT INTO ІТЕМ(ІТЕМ ID) VALUES (3) 


-- Add item 1 & item 2 to Box 1 


INSERT INTO ITEM IN ВОХ(ТТЕМ ID, BOX ID) VALUES (1, 1) -- OK 
INSERT INTO ITEM IN ВОХ(ТТЕМ ID, BOX ID) VALUES (2, 1) -- OK 
-- Add item 3 to both Box 2 & Tray 2 - violating our business rule 
INSERT INTO ITEM IN BOX (ITEM ID, BOX ID) VALUES (3, 2) -- OK 


INSERT INTO ITEM IN TRAY(ITEM ID, TRAY ID) VALUES (3, 2) -- Should FAIL 


* 


(1 row(s) affected) 


(1 row(s) affected) 
Msg 2601, Level 14, State 1, Line 10 
Cannot insert duplicate key row in object 'dbo.ITEM IN TRAY OR BOX V' with 
unique index 'ITEM IN TRAY OR BOX V UI'. 
The statement has been terminated. 
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Бу өе 


How to work around PIVOT's 


column constraint 


QL Server 2005’s introduction of the PIVOT 

statement is a welcomed addition for indi- 

viduals who routinely create reports that work 
with aggregate data because it makes the pivoting 
process simple. However, the PIVOT statement has 
one key limitation: You must know about and include 
all the PIVOT column names when you're writing 
the code. In some cases, this isn’t a major limitation. 
For example, if you're creating a report that contains 
monthly aggregations, the column names will always 
be the month names. However, there are many situ- 
ations that require a more robust, dynamic solution. 
ГЇЇ describe a fairly common business scenario in 
which knowing all the column names isn't possible 
and show you how to create a dynamic PIVOT query 
so that isn't a problem. 


The Scenario 

Suppose you have a database in which instructors 
enter students' names, assignments, and grades— 
and you need to create a report that lists the grades 
students received on their assignments. The code in 
Listing 1 creates and loads a sample data set with 


LISTING 1: Code to Create 
the Data Set 


CREATE TABLE dbo.DemoTable 


AssignmentName VARCHAR(255), 
StudentName VARCHAR(255), 
Grade INT 

) 

GO 


INSERT INTO dbo.DemoTable 
Values('Assignmentl', 'John Smith', 79) 
INSERT INTO dbo.DemoTable 
Values('Assignmentl', 'Jane Smith', 80) 
INSERT INTO dbo.DemoTable 
Values('Assignmentl', 'Paul Smith', 75) 
INSERT INTO dbo.DemoTable 
Values('Assignment2', 'John Smith', 59) 
INSERT INTO dbo.DemoTable 
Values('Assignment2', 'Paul Smith', 65) 
INSERT INTO dbo.DemoTable 
Values('Assignment3', 'Jane Smith', 79) 
GO 
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students, assignments, and grades. (You can down- 
load this listing and the other listings presented here 
by going to www.sqlmag.com, entering 102722 in 
the InstantDoc ID text box, and clicking the 102722 
.zip hotlink.) For simplicity sake, this information 
is loaded into a non-normalized table. However, all 
the examples could easily be expanded to work with 
proper database structures. 
If you execute the command 


SELECT * FROM DemoTable 


against the sample data set, you'll get the grade 
report shown in Figure 1. As you can see, this report 
provides all the necessary information, but 
it's hard to determine how each student is 
doing. It would be much better to create a 
report that looks like the one in Figure 2, 
page 36. This pivoted grade report provides 
the same information but is much easier to 
work with from an end-user perspective. 

Now that you know a pivoted report is the best 
way to present the information, let's look at two 
distinct ways to create it: static pivoting and dynamic 
pivoting. 


Static Pivoting 

If you're familiar with using the PIVOT statement, 
you might be thinking that getting the results in 
Figure 2 is easy to accomplish. I agree that it is 


[r9 
» 


Mitchel Sellers 


(msellers @ iowacomputergurus.com) is the 
CEO of lowaComputerGurus, a Microsoft C# 
MVP, and a Microsoft Certified Professional. He's 
a regular public speaker and the author of 
Professional DotNetNuke Module Programming 
(Wrox) and many technical articles. 


ORE on the WEB 


Download the code at 
InstantDoc ID 102722. 


very easy for this specific 


subset of data. You just AssignmentName era Grade 
-— Assignmentl John Smith 
need to use a script like | Ass ionment] Jane Smith 80 
that in Listing 2, page Frid аенын dec smi br 23 
е Assignment John Smit 
36. This script first uses а | Assignment? Paul Smith 65 
common tableexpresion | Assignment3 Jane Smith 70 


(CTE) to obtain the data 
to be pivoted, then uses a PIVOT query to convert 
the data into the desired format. (If you're unfamiliar 


Figure 1 
The default report 
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DYNAMIC PIVOTING 


statement and QUOTENAME function to 
work correctly. 

After declaring the @Columns variable, 
the code uses the COALESCE statement to 


StudentName Assignmentl Assignment2 Assignment3 
Jane Smith 80 70 
John smith 70 50 
Paul Smith 73 65 
Figure 2 with how static pivoting works, see the article “Create 


The pivoted report 


Pivoted Tables in 3 Steps,” July 2009, InstantDoc ID 
101684.) 

As I mentioned previously, the problem with static 
pivoting is that you need to know about and include 
every column name when you write the code. If the 
database includes a column that you didn’t know 
about, that column won't show up in the pivoted table. 
For example, suppose an instructor adds a column 
named Assignment4 to our sample data set, but 


LISTING 2: Static PIVOT Script 


WITH PivotData AS 
C 


SELECT 
AssignmentName, 
StudentName, 
Grade 

FROM DemoTable 


SELECT 
StudentName, 
Assignmentl, 
Assignment2, 
Assignment3 

FROM PivotData 

PIVOT 


C 

SUM(Grade) 

FOR AssignmentName 

IN (Assignmentl, Assignment2, Assignment3) 
) AS PivotResult 
ORDER BY StudentName 


LISTING 3: Dynamic PIVOT Script 


DECLARE @SQL as VARCHAR(MAX) 
DECLARE @Columns AS VARCHAR(MAX) 


SELECT @Columns = 
COALESCE(@Columns + 
FROM 


', ','') + QUOTENAME (Assi gnmentName) 


SELECT DISTINCT AssignmentName 
FROM DemoTable 

) AS B 

ORDER BY B.AssignmentName 


(B) SET @SQL = ' 
WITH PivotData AS 


SELECT 
AssignmentName, 
StudentName, 
Grade 

FROM DemoTable 

) 


SELECT 

StudentName, 

' + @Columns + ' 
FROM PivotData 
PIVOT 
(6 


SUM(Grade) 
FOR AssignmentName 
IN (' + @Columns + ') 
) AS PivotResult 
ORDER BY StudentName’ 


EXEC (@SQL) 
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doesn’t let you know 
about the additional 
column. When you 
run the static PIVOT 
script in Listing 
2, the information 
about Assignment4 
wouldn't show up in 
the results. 


Dynamic 
Pivoting 
Creating a script that 
will work regardless 
of any new columns 
entered into the 
system might seem 
impossible, but it's 
not. You just need 
to use a two-step 
technique: 

1. Retrieve the 
column names 
and store them in 
a variable. 

2. Build а pivot 
query using the 
variable's values. 


Using this technique, 
Icreated the dynamic 
PIVOT script in 
Listing 3. Let's takea 
detailed look at how 
this script works. 
Callout A in 
Listing 3 highlights 
thecodethat retrieves 
the column names. 
This code relies on 
the COALESCE 


return the first non-null item and concatenate 
the column names as the SELECT statement builds 
the result set. Some people might dislike this approach 
because the @Columns variable is used on both sides 
of the assignment operator (=). However, I like using 
this approach because it’s simple and it works. 

When you perform these types of operations, 
security is extremely important. For that reason, the 
QUOTENAME function is used to get the column 
names. This function returns a safe column name 
that can be used to perform the pivot. For example, 
if someone created a column named myCourse ] for 
fun, a script error wouldn't be generated. Without 
the QUOTENAME function, the script would be 
susceptible to a second-order SQL injection attack, 
where persisted data could be used to create a SQL 
injection attack. Functionality and security are both 
key concerns in this example, which is why it differs 
from many other T-SQL examples that you might 
find on the Internet. 

The code in callout A fills the @Columns variable 
with a list that looks like 


[Assignmentl], [Assignment2], 
[Assignment3], [Assignment4] 


With this list, you can build the dynamic PIVOT 
query. As callout B in Listing 3 shows, the dynamic 
PIVOT query is a modified version of the static 
PIVOT query. Instead of hard-coding the column 
names in the SELECT statement, the dynamic 
PIVOT query uses the names in the @Columns 
variable. After the @Columns variable’s values are 
concatenated in the appropriate spots in the PIVOT 
query, the EXECUTE statement runs the query. The 
results will look like those in Figure 2. 

Note that this example was simplified to better 
demonstrate how to achieve dynamic pivoting. How- 
ever, this technique will work for more complex sce- 
narios as well. You just need to expand the dynamic 
PIVOT query. 


A Valuable and Time-Saving 
Technique 
The two-step dynamic pivoting process is a valuable 
and time-saving technique. Regardless of how many 
columns are added to a database, you'll always get 
accurate results—and those results will be easier for 
end users to work with. Plus, you'll save time because 
you can use this technique for almost any kind of 
situation. ЕР 
InstantDoc ID 102722 
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Time to Defrag Your Indexes? 


There's more to consider than you might think 


question I hear regularly from the SQL Server 
user community is, “How do I know when it’s 
time to defrag my indexes?” Most semi-experienced 
SQL Server DBAs use the avg_fragmentation_in_ 
percent column of the dynamic management func- 
tion (DMF) called sys.dm_db_index_physical_stats to 
determine when an index should be completely rebuilt 
(ALTER INDEX REBUILD), merely defragged 
(ALTER INDEX REORG), or left alone. This column 
reveals the average fragmentation of a given index—the 
higher the value, the worse the fragmentation level is. 
But how much fragmentation is too much? Is it 20 
percent, or maybe 30 percent? Even more? Unfortu- 
nately, there’s no magic number, and the average frag- 
mentation percentage isn’t the only factor to consider 
when determining whether you need to rebuild or reor- 
ganize an index. You also need to consider the fullness 
of the pages, as reported in the avg_page_space_used_ 
in_percent column of the same DMF. There’s more to 
consider than you might think, so in this column ГЇЇ 
discuss some of those considerations, and ГЇЇ provide 
an example and a helpful piece of code that illuminates 
the effect of a fragmented index. 


Be Logical About It 

First, you need to understand two key metrics. The 
fragmentation reported by the avg fragmentation - 
in_percent column shows when the pointer at the leaf 
(1.е.‚ bottom-most) level of an index points to a page 
that’s logically next in line based on the order of the 
index expression but isn’t physically next to it within 
the data file. This situation might occur when a row 
has been inserted into a full page, resulting in a page 
split. Such a split forces half the rows to be moved to 
a new page that most likely isn’t physically next to the 
page that split. The avg_page_space_used_in_percent 
column simply reports how full (on average) the pages 
in the index are. 


What's the Big Deal? 

Now that you know what these metrics mean, you're 
good to go, right? Unfortunately, it's not that simple. 
Yes, in general, the more fragmentation or unused space 
you have on a page, the more likely the system's per- 
formance will suffer. The real question is, “How much 
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is performance really affected?" In reality, 20 percent 
fragmentation might affect performance dramatically or 
negligibly, depending on several factors—including how 
much data you have and how (and how often) you access 
that data. So, there's no magic number or threshold. 
Every situation is unique, so it's crucial that you under- 
stand your system's data-access patterns so that you 
can make the right index-maintenance decisions. This 
is especially true for systems with limited maintenance 
windows and high performance demands. 

Let's see how out-of-order pages (fragmentation) 
can affect performance. First, keep in mind that this 
type of fragmentation has much more effect when 
reading and writing from the physical disk than it does 
with data that’s in SQL Servers memory cache; disk 
access is inherently slower due to physical 
movement of the disk drive heads. For 
example, if you need to read a range of data 
from disk, and the pages aren't physically 
together and in the correct order, the disk 
heads would have to constantly move from 
sector to sector for each fragment instead of gliding 
along in a contiguous movement. Each movement of 
the head costs time and thus affects performance. 

Another factor in a fragmentation scenario is that 
you'll read more pages 
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ORE on the WEB 


See the web listing at 
InstantDoc ID 102460. 


than you would in a non- 
fragmented scenario, 
simply because the data is 
now dispersed more. The 
result is more physical 
and logical I/O and the 
utilization of more CPU 
cycles. (An example later 
in the article clarifies this 
point further.) If your fragmentation results from 
page splits, you typically end up with many pages 
that are less full than if they didn’t split. This is where 
avg_page_space_used_in_percent comes into play. 
Imagine a situation in which you needed to read 
1,000 rows, and they were spread over 20 pages that 
were 100 percent full. If those same pages became 
fragmented, and now the 1,000 rows were spread over 
40 pages that were only 50 percent full, you’d have to 
do twice the amount of work to read the same amount 


time and thus 


Each movement of 
the disk head costs 


affects performance. 
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TIME TO DEFRAG YOUR INDEXES? 


of data. So, the goal is to keep the pages as full as pos- 
sible to reduce the reads—yet to leave enough room 
for inserts and updates to minimize page splits. (This 
is where a proper Fill Factor helps, but that’s a topic 
for another day.) 


Where’s the Catch? 
Both types of fragmentation that Гуе discussed have 
a far greater impact on performance when the indexes 


are used as part of a scanning process than if they're 
used for individual seeks. If you look up any given single 
row via an index, it will take approximately the same 
amount of CPU and I/O regardless of whether the index 
Is fragmented. You need to read only a small number of 
pages for a seek, and the cost will be the same for any 
given row. Most well tuned online transaction processing 
(OLTP) queries work this way. However, both types of 
fragmentation can significantly hinder a full or partial 


index scan. For clustered indexes, a partial or range scan 
is common for queries that search for a range of values 
in the WHERE clause. In this case, the extra reads 
due to the out-of-order and less full pages can add up 
quickly, as the following example shows. 


Example 

For my example, Гуе built a table with 10,000 rows 
with about 12 rows to a page. It has a clustered 
index on an integer column, with values ranging 
from 1 to 10,000, and a non-clustered primary key 
on the integer Identity column. 

If you run the code in Web Listing 1 (www 
.sqlmag.com, InstantDoc ID 102460) down to step 
1, you can see that it takes 21 reads to read the range 
of values between 499 and 700 and three reads for 
the singular value of 500, with the clustered index 
having 834 total pages. Now, run to step 2, and 
you can see that a single-row insert caused a page 
split, as witnessed by the total pages increasing by 
one, and another read for the range scan has been 
added. Next, 10 more rows are inserted, increasing 
the page count by 10. But the range scan reads have 
increased to 30, a difference of over 30 percent, yet 
the singular query stayed at three reads. 

You can see how just a few changes can affect 
partial scans in a dramatic way while leaving the 
seek virtually unaffected. If you try this with a Fill 
Factor of 80 percent for the clustered index, you 
can virtually eliminate the page splits, but you get 
1,000 total pages instead of the original 834. The 
result is increased sizes of indexes on disk and less 
fitting into cache. 
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SSIS into a 
Data Quality Toolkit 


Plug these data quality transforms 
into your SSIS pipeline 


Profile your data 


Cleanse your data 


Parse & standardize your 
contact data 


Match/dedupe records 
across data sets 


Enrich your data 
It's Up to You 


It's up to you to decide when the extra CPU and 
I/O caused by fragmentation become problematic 
in your application. Usually, increased CPU con- 
sumption and longer durations are the first signs 
of trouble. Larger indexes generally fragment faster, 
and indexes smaller than several hundred pages can 
often be ignored. Due in part to their increased size, 
clustered indexes might need more attention than 
non-clustered ones, too. But usage is also a key 
factor in deciding when to deal with fragmentation. 
En 
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Left-Brain.com is the newly launched online superstore stocked with 
educational, training, and career-development materials focused on 
meeting the needs of SQL Server professionals like you. 


SOL Server 2008 System Views Featured Product: 
= I SQL Server 2008 System Views Poster 


Face the migration learning curve head on with the SQL Server 
2008 System Views poster. An updated full-color print diagram of 
catalog views, dynamic management views, tables, and objects for 
SQL Server 2008 (including relationship types and object scope), 
this poster is a must-have for every SOL DBA migrating to or al- 
ready working with SQL Server 2008. 


Order your full-size, print copy 
today for only $14.95*! 


*Plus shipping and applicable tax. 


brai 
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SimplelPC 1.0 


тайшопа .NET Framework applications com- 

municate over the database located in the same 
system or over a LAN. But .NET remoting applica- 
tions, such as distributed transactional applications 
that include SQL Server, share and exchange data 
across networks in a distributed client/server architec- 
ture. Such applications need to include some type of 
fault-tolerance mechanism to accommodate applica- 
tion errors that may occur during the communication 
process. SimpleIPC, developed by Dallas-based Cogni- 
tier, isolates all exceptions to a separate process without 
causing any interruption to the main project. 


SimpleIPC Overview 

The primary function of SimpleIPC is to keep unhan- 
dled exceptions and subsequent process crashes out of 
your main web application process. This mechanism 
will help your application run without interruption by 
repeating the call against a different external process, 
thus causing no impact for your end users. To work 
with the product, developers should write server and 
client code separately. Although the server code is used 
to run inside the IPC server, the client code is incor- 
porated into your client application for exchanging 
information with the IPC server, which can either be a 
Windows Forms project or an ASP.NET page. 

The server routines are written using either C# or 
Visual Basic and consist mainly of functions, which 
form the core part of your application, as the resulting 
client project makes a call to the functions mentioned 
in the server routine. You just need to establish refer- 
ences to certain DLLs included with the product and 
implement certain interfaces if you have Visual Studio 
2008. If you don’t have Visual Studio, you can use 
the GUI-based script editor utility included with the 
product, which Figure 1 shows, to author and compile 
DLLs in JScript. NET as well as templates that auto- 
matically add references to your project. 

In addition to providing the editor, SimpleIPC also 
lets you tweak system, server, and logic settings and 
management of routines and events, as Figure 2, page 
42, shows. The product also supports advanced mecha- 
nisms, such as throttling, caching, and object pooling. 
These features are meant to make applications secure 
and enable them to load faster and more efficiently. 


Monitoring .NET Remoting 

SimpleIPC smartly acts like a middle layer on top 
of .NET remoting that monitors the output of your 
application via reflection at runtime. Exceptions are 
isolated in IPC Server, so that your main application 


SQL Server Magazine * www.sqlmag.com 


will run without any interruption. As a developer, you 
have to register your custom code modules at design 
time, and the product analyzes your DLL to determine 
the class names. 

While the client code passes through .NET, the 
additional layers accommodate clients using Java, 
COM, or web services. Moreover, the permission to 
activate and access servers is controlled via member- 
ship in designated local computer groups, which 
could enhance security. As Figure 3, page 42, shows, 
the product also enables you to define inputs and run 
tests with a configurable number of simulated clients 
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р "=" шаға 
Fsmpe PC) camry 
File Нер 
System Settings | Server Settings | Logic Settings | Manage Routines | Manage Events | Author Routines | Test Routines | Server Status | 
Source File HelloWorld bat x: Reference Name Or File Name 
C:\Program Files\Cognitier\SimplelPC\Development\Bin\CTCommon. dil 
В " C:\Program Files\Cognitien\SimplelPC\Development\Bin\CTSHost.dil 
Compiled Binary Не 5 
system 
{New Source File | | Delete Source File | 
{Add Reference | [Delete Reference 
Browse for references starting from: 
[C:AWindows\icrosoft. NETFrameworkiv2.0.50727 М 


import System; 
import System.Collections; 
import System.Reflection; 


[assembly-AssemblyVersion("1.0.0.0")] 


public class HelloWorld implements CTSHost.IBasicRoutineRun1 
{ 


var oLogUtil: CTCommon.LogUtil; 
var bRet: System Boolean: 


oLogUtil = CTCommon.Logutil.Instance;] 
bRet - false 
try 
t 
/[Examine input arguments - always Strings 
var sValue: System.String; 
var iCount: System Int32: 
iCount = 0; 
while (iCount < oCallParamsAccessor InputArgs.Count) 


il m 


public function RunRoutine(oServerContextAccessor: CTSHost.ServerContextAccessor, oSessionContextAccessor: CTSHost.Sessior 


m 


Line 15 [ {Save And Compile | 


Revert | 


Figure | 


Using the editor to author and compile DLLs in JScript. NET 


SIMPLEIPC 1.0 


Pros: Provides various settings to let developers stabilize distributed 
application performance; allows application testing using simulations 


Cons: Documentation is a bit hard to understand 


Rating: Yi Yr ke we УС 


Price: $299 


Recommendation: If you're an experienced developer of distributed .NET 
applications and want to add fault tolerance to your apps, SimplelPC is worth 


checking out. 


Contact: Cognitier * www.cognitier.com 
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SIMPLEIPC 1.0 


File Help 
System Settings | Server Settings | Logic Settings | Manage Routines | Manage Events | Author Routines | Test Routines | Server Status 


Instance 


Application 


On Server Start 


On Server End 


On Session Start 


On Session End 


Before Routine Invoke 


After Routine Invoke 


Figure 2 


Adjusting settings to improve application performance 


File Help 
| System Settings | Server Settings | Logic Settings | Manage Routines | Manage Events | Author Routines | Test Routines | Server Status 


Test Cases 
Test Case [Default М (LNewTestCase | [Delete Test Case | 


Test Case Routine Input Parameters 


ЕЕ Input Parameter 
1234 


Routine To Call 


Entry Point |Out Of Proc COM ~| #0fThreads/Sessions 1 Е Single Call ж of Calls Per Session 1 НІ 


Java Home Directory C:\Program FilesJavaljre1.5.0 11 


Java Cmd Line Args — -Djava library.path="C:\Program Files\Cognitier\SimplelPC\Bin" -cp "..%CLASSPATH%:C:\Program Files\Cog 


Session Output 


Figure 3 


Defining parameters for testing applications 


memory consumption. I 
expect that this feature 
will be highly useful for 
resource-intensive appli- 
cations. Moreover, devel- 
opers will be able to use 
SimpleIPC to call the 
server code in a Java client 
application via COM. The 
SimpleIPC team has pro- 
vided an article at cog- 
nitier.com/downloads/ 
ServletWordDocSample 
.pdf that examines this 
concept comprehensively. 

SimpleIPC includes 
comprehensive documen- 
tation, which you can 
download at www.cogni 
tier.com/All-Docs_ep_66 
-html. I found it little dif- 
ficult to understand the 
content, but the support 
team provided excellent 
assistance within one busi- 
ness day. Cognitier also 
provides exercises and 
articles that include step- 
by-step solutions to repro- 
duce procedures, along 
with complete source code 
and screenshots. I don’t 
recommend SimpleIPC 
for beginners because 
they wouldn’t use it in 
the scenarios for which 
it was developed. I sug- 
gest that the vendor create 
exercises in video format 
to help developers more 
easily understand how the 
product works. 


Error 
Prevention 
Tool 

SimpleIPC should be 
useful for developers who 
are working on advanced 
distributed projects in 
which runtime errors can 
be fatal for their applica- 


making a configurable number of invocations, each tions’ performance and stability. If you don’t want 
through various programming APIs. unforeseen events to occur during an application’s life 
The product also recycles frequently in case your cycle, I suggest you give SimpleIPC a try. En 


code has memory leakage, thus letting you control 
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We would never tell a lie... 


That's why we're going to let our readers 
tell you why SQL Server Magazine is the 
top independent publication and Web 
site in the IT industry. 
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Download Central brings you the tools to meet your most critical SQL needs. 


A one-stop hub of countless free trial downloads from leading 
industry vendors, Download Central has done all the looking. 


All you have to do is see which tool is the best fit. 
And you get to do it all for FREE! 


Score Your Solution at Download Central! 
sqlmag.com/go/downloads 
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Bytes from the Blog 


www.sqdlmag.com/go/industrybytes 


Free Tool Addresses SQL Server 
Password Vulnerability 

hen a vulnerability is found in a Microsoft 

product, you expect it to be patched as quickly 
as possible. However, a password exposure vulner- 
ability that was found in SQL Server a year ago has yet 
to be patched. In fact, Microsoft has no plans to patch 
the security problem. 

A member of Sentrigo’s Red Team, a group of 
security experts who focus on uncovering security 
problems in database applications, discovered that 
passwords were being kept unencrypted in memory for 
specific types of connections in SQL Server. Therefore, 
anyone with administrator rights can see not only the 
passwords of other SQL Server users, but also the cre- 
dentials used by applications that access the server with 
SQL Server authentication. Because most users use 
the same password for multiple applications, a rogue 
administrator or an attacker that hacked into the SQL 
Server system could potentially use these passwords 
to access more than just SQL Server applications. The 
vulnerability is found in SQL Server 2008, 2005, and 
2000 systems that are running on Windows OSs and 
using mixed authentication mode. 

“Tt’s security 101: Never have a clear text password 
anywhere,” Sentrigo CTO Slavik Markovich told me. 

After discovering the vulnerability, Sentrigo’s 
researchers contacted Microsoft's Research Center in 
September of 2008. However, Microsoft responded to 
the issue by saying that the problem didn’t look like a 
security issue because you need administrative privi- 
leges to exploit it. 

“Microsoft has thoroughly investigated claims of 
vulnerabilities in SQL Server and found that these 
are not product vulnerabilities requiring Microsoft 
to issue a security update,” said a Microsoft spokes- 
person. “As mentioned by the security researchers, 
in the scenario in question, an attacker would need 
administrative rights on the target system. An attacker 
who has administrative rights already has complete 
control of the system and can install programs; view, 
change, or delete data; or create new accounts with full 
user rights.” 

Sentrigo has released a free tool called Password- 
izer that addresses this security risk by erasing the 
unencrypted passwords. You can also set the tool up to 
remove passwords from memory automatically in the 
future. To download Passwordizer, visit www.sentrigo 
.com/passwords. SOL! 
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asp.netPRO to Relaunch as 
DevConnections 

enton Media is relaunching its web development- 

focused magazine, asp.netPRO, as DevConnec- 
tions magazine. The new magazine will continue to 
provide how-to articles on building enterprise ASP 
.NET web applications, but the new DevConnections 
magazine will have broader coverage of the .NET 
Framework, Visual Studio, and beyond. 

Two key factors driving this change are alignment 
with the 2010 wave of Microsoft developer releases 
and the changes they will bring to web and application 
development, and alignment with the Penton Media's 
DevConnections conferences. The new magazine 
will cover .NET Framework technologies, such as 
Silverlight and Windows Communication Foundation 
(WCF). It will also cover Visual Studio and database 
programming. There will be interviews with experts 
in the development community, and a column that 
focuses directly on insider information at Microsoft. In 
addition there will be a monthly guest editorial written 
by an industry expert who will work with magazine 
staffers to pull together cutting-edge articles that drill 
down on a specific technology, such as Windows Pre- 
sentation Foundation (WPF), the Entity Framework, 
or AJAX. 

Along with expanded coverage, expect to see a 
larger magazine. Although there will be several articles 
focused on a specific technology, the magazine will also 
continue to cover a broad spectrum of technologies to 
give readers lots of choices every month. 

Developers know that changes are coming with 
Visual Studio 2010 and .NET Framework 4.0. Michele 
Bustamante and other authors have already begun to 
cover the .NET Framework in depth. Other authors, 
such as Dino Esposito, have already begun to cover 
Visual Studio; look for more articles on the Visual 
Basic and C£ languages and the Visual Studio Team 
System. 

Many developers know about the DevConnections 
conferences. What you may not know is that DevCon- 
nections magazine and the Connections conferences are 
both part of Penton Media. By joining forces, it ensures 
that readers will see the same high-quality technical 
content in the magazine that the conferences deliver. 

DevConnections magazine will be distributed this 
Fall at the DevConnections conference November 9-12 
at Mandalay Bay in Las Vegas, and at the Microsoft 
Professional Developers Conference (PDC) November 
17-20 in Los Angeles. SQL] 
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BACKUP AND RECOVERY @ Editor's Tip 
Back Up Applications Locally or to the Cloud 


AppAssure has released the latest version of its backup and recovery platform, Replay 4. This server-based, disk- Got a great 
imaging product lets you back up your applications locally or to the cloud. It also offers continuous application — new product? 
protection technology, which lets you quickly recover from disasters and automatically test restore. According бөлі announce- 
to AppAssure, “Our unique approach to backup combines the best features of the top imaging, replication, and ^ ments to products @ 
deduplication software on the market today, offering a 3-for-1 solution that guarantees continuous recovery- 
testing, seamless disaster recovery, always-on high availability, and significantly reduced storage costs." This ver- 
sion supports SQL Server 2000 and later, as well as Windows 7, Windows Server 2008 R2, VMware ESX 4, and 
Hyper-V R2. You can download a trial version of Replay for SQL from www.appassure.com. 


sqlmag.com. 
—Brian Reinholz, 
Production Editor 


BUSINESS INTELLIGENCE 

Export Analytical Findings with | Click 

Future Point Systems recently released new versions of its Starlight 
Visual Information System (VIS) and Starlight XML Engineering 
Environment (XEE). According to the vendor, the new versions include 
overall improvements to the products' workflow, namely in importing 
data into the system for analysis. Starlight 4.1.1 now supports one-step 
import of PDF documents, along with Microsoft Office documents, 
including Excel, Power Point, and Word. Starlight 4.1.1 also offers 
drag and drop functionality, which helps reduce the time it takes users 
to set up analyses. Another key feature in this release is one-click web 
reporting, which you can use to share information with many users in 
multiple formats. For more information about the Starlight software 
suite, visit www.futurepointsystems.com. 


DATABASE SECURITY 

Real-Time Database Activity Monitoring and Auditing 

Database security software vendor Sentrigo announced that the new version of its database activity monitoring 
software, Hedgehog 3.0, is now available. Hedgehog 3.0 helps you protect your data—and prove to compliance 
regulators that you are doing so—by offering data masking based on customizable filters, high-performance 
auditing that lets you log database activity to a flat file, and monitoring of excessive behavior that could be 
indicative of a security threat. In addition to the new Hedgehog 3.0 features, Sentrigo's Hedgehog vPatch now 
provides additional SQL Server-specific protections. This version supports all versions of SQL Server. You can 
learn more about Sentrigo's Hedgehog family of products at www.sentrigo.com. 


BUSINESS INTELLIGENCE 
Uncover Data that Charts and Spreadsheets Can't Expose 
New data-mapping technology is now available from — 

> Store and Customer Viewer < е бендове Purchases by PIP Cade 
geographic information systems (GIS) vendor ESRI. беу БУД me 
Maplt is a software plus services solution that lets 
you create maps that display your business data. With 
Maplt, you can uncover patterns and trends in your 
data, as well as better analyze your data (such as 
customer demographics) in a geospatial context. The 
data is pulled from SQL Server 2008 or Microsoft 
Excel, and you have the option to display maps in 
Microsoft Silverlight, Windows Presentation Foun- 
dation (WPF), or Microsoft SharePoint Server 2007. 
You can download a free 60-day trial of Maplt from 
www.esrl.com/software/mapit/index.html. Ей 
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he move to the cloud has been a big trend for 

technology vendors this year, but for businesses, 
moving to the cloud entails making many application- 
related changes. Even so, the cloud promises scalability 
and cost savings that make it worth considering. The 
cloud in this case is SQL Azure, formerly SQL Data 
Services, Microsoft’s SQL Server-based database cloud 
offering. Here are seven facts about SQL Azure. 


SQL Azure is based on SQL 
Server. 

Using SQL Azure is much like using an on-premise 
version of SQL Server. However, SQL Azure initially 
offers only a subset of SQL Server's features. SQL 
Azure will provide basic relational database capabilities 
but initially won't offer BI functionality. 


SQL Azure runs on Microsoft 
hardware. 

Servers hosted in Microsoft’s data centers provide the 
SQL Azure services. Databases you create on these servers 
are available on a global scale. Microsoft personnel per- 
form the hardware operations and management. 


SQL Azure supports T-SQL. 

SQL Azure supports most T-SQL statements as well 
as a full range of DML and DDL commands. It 
doesn’t support commands that affect the underlying 
hardware, such as the resource governor. It also sup- 
ports most SQL Server data types including bigint, bit, 
decimal, int, money, numeric, smallint, smallmoney, 


6 7 Facts About SQL Azure 


tinyint and char, varchar, but not the LOB data types 
and not the newer geo-spatial data types. 


SQL Azure uses TDS. 

Just like on-premise SQL Server, SQL Azure supports 
client connections using the native SQL Server tabular 
data stream (TDS). You can create SQL Azure appli- 
cations using a variety of development tools including 
Visual Studio, and you can use such middleware as 
ODBC, OLE DB, ADO.NET, or PHP. SQL Azure 
uses SQL Server authentication, not Windows. 


SQL Azure is a service. 

SQL Azure can’t access hardware, so you can’t do 
backup. Instead, you must use a data copy function 
like BCP. Other things that aren’t available include 
using sp_configure to change server options, and SQL 
Profiler. 


SQL Azure is in two editions. 
SQL Azure Web Edition will support a maximum of 
1GB of data and will be priced at $9.99 per month. 
SQL Azure Business edition will include up to 10GB 
of data and will cost $99.99 per month. 


SQL Azure is going live. 

You can find the SQL Azure CTP at the MSDN 

site (msdn.microsoft.com/en-us/sqlserver/dataservices/ 

default.aspx) in November. SQL] 
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Video: SharePoint 
Virtualized with Hyper-V 


Learn how a dynamically provisioned 
data center can help achieve high 
availability in a virtualized SharePoint 
environment. In this brief video con- 
figuration scenario, you'll learn how 

the F5 Management Pack works with 
Microsoft System Center technologies 

to monitor SharePoint traffic and the 
whole virtualized, environment, and then 
take appropriate action to maintain high 
performance and availability. 
windowsitpro/com/go/ 
VirtualizedSharePoint 


Savvy 
Assistants 


Your guide to sponsored resources 


Follow us on Twitter at www.twitter.com/SavvyAsst. 


Meeting Compliance 
Objectives in SharePoint 
In recent years, the business and political 
landscape has seen incredible change 

with regard to the rules and regulations 
governing the stewardship of electronically 
stored and processed information. Compli- 
ance has become critical. This white paper 
aids IT administrators —and other stake- 
holders responsible for managing Microsoft 
SharePoint deployments—in planning and 
implementing a comprehensive, reliable, 
and efficient compliance strategy appro- 
priate to their organizational needs. 
Windowsitpro.com/go/ 
SharePointCompliance 


Exchange Server 2010: 
Deploying Unified 
Communications— 

Free online event 

Learn how to turn your Exchange 2010 
deployment into a launch pad to the United 
Communications future! Join expert Paul 
Robichaux on Dec. 1 as he presents a clear, 
insightful, and independent look at how 
your Exchange deployment can help you net 
the benefits of Unified Communications. 
windowsitpro.com/go/ 


| ExchangeServer20 10DeployingllC 
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SOC SENTRY 


PerformanceAovisor ° 
for SQL Server 


SQL Server + Windows 
Performance Dashboard 


Real-Time & Historical 
Performance Analysis 


Disk Activity, Latency, 
and Capacity Monitoring 


Top SQL Analysis Highlights 
Heaviest Queries 


Graphical Blocking and 
Deadlock Analysis 


Free Trial Download at: sqlsentry.net/sql-performance 


performance solution 
for Microsoft SQL Server 


C! SENTRY - 
PerformanceAoVvisor ? 


for Analysis Services 


- Powerful SSAS Performance 


Dashboard 


- Innovative Workload and 


Bottleneck Profiling 


* Capture of all Heavy MDX, 


XMLA and DMX 


- Aggregation, Partition and 


Dimension Activity by Query 


• SSAS Cache and Storage 


System Monitoring 


cO SENTRY 
Event/Vlanager ° 


e Calendar Views of Top SQL, 
Blocks, Deadlocks, Jobs 
and other Events 


* Calendar Views Combining 
SSAS, SQL Server, SSIS, and 
SSRS Events 


e Easy Job Chaining 
and Queuing 


e Robust Alerting and 
Response System 


PE З 


PATIENCE IS А 


WE DISAGREE! 
, 


EVER HAD TO RESTORE A BACKUP FILE JUST 
TO PERFORM A 5-MINUTE TASK? WHAT A PAIN! 


Why wait? you can 
access data and objects in backups files IMMEDIATELY, 
without waiting for a restore. 


IT'S AS EASY AS 1, 2, 3: 

1) Browse your backup files to find the one you need— full, differential, and log! 

2) Use SQL virtual database to attach the backup file to SQL Server— IN SECONDS! 

3) Use ANY native SQL Server or third party tool to query and extract the data you need—IMMEDIATELY! 


STOP WAITING! 
GET INSTANT GRATIFICATION WITH SQL VIRTUAL DATABASE. 


DOWNLOAD NOW: 


